All pastors are to be apt to teach - therefore they would be included in what James was speaking about, as would all other teachers.

Though I do believe Ephesians refers to five different gifts given to the church: apostles and prophets (who laid the foundation for the church), evangelists (who travel around, like some kind of missionary), pastors and teachers (the "and" in both the Greek and English indicating there are two types of people referred to here - that are given to each church, not just one).