Good point Rik, we actually had a similar problem with same USB driver... turns out they were using a ADSL USB modem, and USB Keyboard/mouse and USB printer/Scanner etc etc... It turns out the inbuilt USB Controllers have a max power limit of ~500mA, and most modems pull around ~450mA then your other peripherials...

Solution was to get a ADSL modem with Ethernet connection, but perphaps you could get away with trying out a POWERED USB hub