The Temporal Lobe!
The temporal lobes are located in the areas around the right and left ears. Not surprisingly, they deal with sound recognition—everything from noise to music to speech. The temporal lobe and the adjacent area is where the famous Broca’s area and Wernicke’s areas exist, although typical language recognition and usage is usually mostly only relevant on the left side in most humans. The right side of the brain is typically more 'musical'. Because of this, it does play a role in communication too! It is important for understanding things such as prosody. (In rare cases, the right and left side functionality is switched around to the opposite side.)

Broca’s area, just outside the temporal lobe, is associated with speech reproduction (speaking skills), while Wernicke’s area (within the temporal lobe) is associated with speech comprehension (understanding skills). Damage to those two areas produce speech and comprehension impairments, respectively.

Again, without the PFC these skills are not very useful, so it must be stressed that the temporal lobes alone are not responsible for listening and speaking skills. The PFC and various other areas of the brain are significantly involved in the processing and development of listening and speaking skills.
Stacks Image 915
Stacks Image 909
Stacks Image 1416