You can listen to anecdotes about how the missile launcher is or is not effective, but look at the numbers. Fully upgraded to hit as hard as possible, and even including a 4th level passive power damage and weight capacity bonus, the missile launcher hits at about 420 damage, with 50% more effect against armor. That in itself is not bad. It's like incinerate without a damage over time effect. The problem is, the missile does not hit nearly as reliably. Again, people will say that it does, and people will say that it doesn't. In my experience, it's quite difficult to get it to hit, so much so, that if you're re-positioning yourself in order to make sure your missile launcher hits stuff, you're probably thinking too much about hitting stuff with it. It's just not worth it.
Plus you factor in that if you want to put points into missile launcher, you have to take away from something else that is infinitely more effective. Do you take points from fitness? Because the Destroyer can be extremely tanky. With an upgrade to shield health and shield recharge in devastator mode and full shields, you're fully capable of withstanding the long periods of damage due to exposure from firing sustained bursts. Do you take points from multi-frags? I guess you could, but this power is a much more reliable killer of basic troopers and it also does a good job of staggering just about everything else. Do you take points from the passive tree? Again, I guess you could, but you can really max out your weapon damage potential here. I'm not even gonna discuss not maxing out devastator mode.
To sum up, the Destroyer is a class that is a true soldier. Tough and versatile, your primary focus should be to use your weapon to kill things, with multi-frags as a crowd control device or a panic button. The missile launcher is super cool looking, and I give kudos to Bioware for adding this neat feature. I just wish it worked better.