Indeed it should be added that recent research has shown that a few little known languages—Kettish, the extinct Kottish, and two or more—which were spoken on the upper Yenisei River in the immediate vicinity of Finno-Ugrian languages and were originally supposed to belong to that family, are in fact not related to it, or apparently any other language, and form a little family of their own.
Source: wiktionary
On the other hand, as Ligeti (1950) was the first to point out, there is a good possibility that they may have spoken a language belonging to the Palaeo-Siberian family, the only surviving member of which is Kettish, also known as Yenissei-Ostyak.
Source: wiktionary
His tenative suggestion is that the Xiongnu language belonged to the Paleo-Siberian family, now represented only by Kettish, or Yenissei-Ostyak.
Source: wiktionary