Do I seem like a ****** if I don't want to hang out with my coworkers?

This one guy acted so dejected. They talk about how they drove around, drank, partied, and smoked pot on the job. I don't really like any of those things, especially with people I don't really know that well. I don't want to seem snooty, but I just don't think I'd have a good time.