My mechanic (karl, zeit00), has said he's driven a lot of e46 m3's, and he also kinda expected more. I mean, don't get me wrong, they're amazing cars and I'd trade mine for one in a second, however he said e36 m3's are more of a fun driver's car... while e46 m3's have more style and comfort also.
I personally would take the e46 over e36 no questions though. hah