Octavian Ursu of the center-right Christian Democratic Union (CDU), Germany’s main governing party, won more than 55% of the vote in the city of Görlitz.
AfD candidate Sebastian Wippel, 36, was deemed the frontrunner after he won the first round of voting.
The vote had been viewed as a litmus test for his anti-immigration party.
The AfD registered high levels of support in Görlitz, which has seen a huge journey of younger people due to a lack of employment opportunities.
The area around the town has served as the backdrop to major Hollywood films, such as Inglourious Basterds and The Grand Budapest Hotel. Ahead of the vote, a number of actors and filmmakers linked with the city wrote an open letter advising residents not to support the AfD.
The party’s initial success led to speculation that it could make an electoral breakthrough ahead of regional elections in September.
But the result ultimately provided a boost to Chancellor Angela Merkel, whose governing coalition with the Social Democratic Party (SDP) has been shaken by heavy losses in the European elections in May.
“I am happy that a majority has chosen to vote for me,” Mr. Ursu, who moved to Germany from Romania in the 1990s, said following his victory. “But in the end, it is not about two candidates but the orientation of this town to the outside world.”
“We live in an open society and do not isolate ourselves,” the 51-year-old said.
Mr. Wippel – a former police officer – said the AfD remained in a “good position” despite the loss.
“It was not a vote for Mr. Ursu but more a vote against me,” he said. “The CDU had to rely on support from many groups… without whom they would not have made it.”
The AfD entered the federal parliament for the first time in 2017 after it won 94 seats in the 709-seat lower house (Bundestag).