I haven't actually tried that version, although I do have A-Shell 5.1.1134 and ATSD 5.1.106.0 running locally under Vista Business x64, with no obvious problems. The ATSD update to 5.1.106 did address some Vista-related complications, so I would definitely recommend that you have at least that version, and I wouldn't be surprised if 2008 Server made it slightly harder to install the ATSD service automatically, although I would imagine that you could overcome any such issues if necessary through the Services applet.
The main x64 issue is the installer itself - for versions prior to 5.1.1145, you need definitely need a "-sb" version of it. After that, we dropped the -sb and all of the install packages are x64 compatible. If you haven't already read the
Vista x64 Compatibility thread, definitely do that. (Note, however, that the somewhat negative comments about 64 bit in general at the end are targeted at workstations, not servers; a large server could probably benefit significantly from x64, especially by allowing for >4GB of RAM.)
If you're migrating from Linux, I would still be surprised if the Windows server wasn't significantly slower than the Linux server, although maybe the newer hardware will be able to overcome the OS efficiency differential. In any case, using ATSD/telnet rather than file-sharing should eliminate the major Windows bottleneck (which is file-sharing).
