First, let me admit that I don't know you. Therefore, there are many questions that should be answered, before answering yours. "Are you a believer in jesus Christ as your personal Savior, Lord, and God?" is one of them. "Where do you fellowship with other christians?" is another one.
A very general answer to your question will be: Our Lord was with the Tax Gatherers and people of low esteem, because He came to save them. We are to preach the gospel of salvation to such. That may need certain friendship. If you are living in this world, then that may require certain friendship with unbelievers. Paul had this kind of friendship, especially when he was chained to Roman soldiers. Many came to know the Lord as their Savior.
Having said these, let me ask, do you preach the gospel of salvation to your unbelieving friends? Or, do you just go along with them, to be conformed to this world? The later is not scriptural. You have to maintain your testimony as a child of God, if you are one.