Religion in the Alternate Realms is very similar to the Forgotten Realms. The world is split into a variety of regional and racial pantheons that represent which group of religions hold the most sway in a particular geographic or racial grouping.
This is only true for members of nonhuman races living in and around the continent of Faerun. Those living in the Hordelands, Kara-Tur, Maztica, etc, may worship a very different pantheon of gods (and then again they may not).
Racial Pantheons: The racial pantheons represent the various races other than humans which have a set of religions that are worshipped by the entire race as a whole. This includes the Elven Pantheon, the Dwarven Pantheon, the Gnomish Pantheon, the Halfling Pantheon, even the Orcish and Goblin Pantheons.
Regional Pantheons: Regional Pantheons only really applies to the human race which is still in a state of religious infancy, with the various human sub groupings only beginning to mingle within the last thousand years. As a result of this a number of regional pantheons have slowly coalesced over time to form the Faerunian Pantheon, but this emerging super pantheon is yet to absorb the remaining Untheric and Mulhorandi Pantheons to become a single racial pantheon for all humans in Faerun.
A church in the Alternate Realms is not analogous to the church building of the real world. Instead a church refers to a religious organisation where all the members have the same or similar beliefs and all obey a single authority. In some instances a church can refer to a single building that houses such a religious organisation but it is also just as possible for a church to have multiple buildings and thousands of members spread across many regions.
In general the religion of a single deity is made up of multiple churches spread across Faerun, and more than a few of these churches actively (and sometimes violently) compete with one another. Very few religions are comprised of a single church and schisms happen regularly with new churches appearing and disappearing.