When in Real Band, I try to assign a vst instrument to a track. Let's say I assign BIAB's Jazz Organ VST to track 1. Then I assign BIAB's 12 String guitar to track 2. Check back on track 1 and the 12 String guitar is selected there as well. What an I missing?

To assign a completely different VSTi synth to a different track you would need to load the VSTi synth into a different Dxi/VSTi Synth effects group in the DirectX/VST Window, and then set the output port of the 2nd track to the port that you loaded the different VSTi into. (right click on the track and set the port)

Think of it like this
Track one is using the VST for Organ
If you assign the same instance of that synth then set it to Guitar, that instance of the synth is now assigned to guitar.

It is possible to create another separate instance of the synth by using the Port option Jeff mentions.

Unless your VST is a multitimbral synth, that is how it works.
If it is a multitimbral synth and can use different sounds on different MIDI channels, then you could try assigning MIDI channels, but you didn't mention which particular VST Synth (VSTi) you are using, so it's hard to get more specific.