The Barcelona-based fashion designer Custo is under fire for plans to build a hotel on a beauty spot in Costa Brava.

Campaigners led by two groups – Salvem l’Empordà (Save Emporda, the region Cadaqués is in) and SOS Costa Brava – protested on Friday outside a Custo shop in Barcelona demanding that work be stopped on the project to build the 4,000 sq m Hotel Custo and 104 houses on land in Sa Guarda in Cadaqués.

Sa Guarda is a Unesco world heritage site and Cadaqués is known for being one of the most unspoilt villages on the Costa Brava and for its association with the surrealist artist Salvador Dalí, whose summer residence was at nearby Port Lligat.

Custo Dalmau and his brother, David, have owned the 150,000 sq m site for several years and were originally granted planning permission in 2010 but work stopped when the regional Catalan government introduced a moratorium on new building in the region, part of which is the Cap de Creus national park.

Although the dispute has rumbled on for years, things came to a head few days ago when work started on an access road to the site, which protesters claim will involve uprooting 250 olive trees and destroying ancient dry-stone walls.

Marta Ball-Llosera, spokeswoman for Salvem l’Empordà, said the group was taken by surprise when road-making machinery suddenly began working on the site.

The legal situation was complicated, she said, because the moratorium extended to building on the site but did not cover urbanisation such as road building and installing street lights, for which the Custos have permission.

“We don’t need another hotel,” said Lluís Castillo of SOS Costa Brava. “As it is, last year the water treatment facilities couldn’t cope with the waste water and it overflowed. It was disgusting. There isn’t capacity for more.

“We are asking the Custos to invest in our natural and cultural heritage rather than urban speculation.”

Earlier this week, the Catalan parliament said it was up to the local authorities to rule on developments such as Sa Guarda. Meanwhile, the regional government is expected to rule next week on whether the works can go ahead.

“The local authority and the Catalan government keep passing the buck,” says Castillo. “But neither of them has done anything.”

The narrow streets of Cadaqués are already overwhelmed by visitors. In summer, hundreds of cars trying to enter the village form lengthy tailbacks.

Over the years many artists and writers have been drawn to Cadaqués, among them Dalí, Pablo Picasso, Joan Miró, Marcel Duchamp, René Magritte, Man Ray and Richard Hamilton. Other visitors include André Breton, John Cage and Walt Disney.

The dispute has echoes of the conflict between former El Bulli chef Ferran Adrià and locals when he sought permission to expand the former restaurant site at Cala Montjoi on Cap de Creus to set up his El Bulli Foundation.

The scheme was originally rejected by the local authority on grounds that it conflicted with zoning regulations in the national park but they relented when Adrià threatened to take his foundation to the US.

If the hotel ever gets built, Custo will join fashion designer Armani and shoemaker Camper who have already diversified into the hotel business. No one from the Custo organisation was available for comment.