Not having much experience with porting heads personally, but taking a few classes on flow, here is what I learned:
Wall roughness does matter, just not a ton. Obviously, the smoother, the higher the velocity, and the more laminar your flow can be.
Laminar on top, Turbulent on bottom
Cross sectional area does matter, but also, not as much as you might think. The way you approach a decreasing area makes more of a difference in pressure drop. A nice G0 curve is best, to a taper which is still okay, to a ledge which is very bad.
I have also heard that a plenum before the valve helps, so a big bowl. I have seen old school hot rod guys actually port through the head and then weld in filler and them smooth it out so that they can get a bigger plenum at the valve.
My thoughts on velocity would be, not to worry too much about slowing your intake velocities if you are smoothing your walls. It'll probably even out to have similar velocities to OEM velocities. And if you are building your own intake manifold, look into velocity stacks in the manifold. That helps velocities like CRAZY.