The gray seal (Halichoerus grypus) population in the Baltic Sea is flourishing. On the one hand, this can be interpreted as successful conservation management but, on the other, the gray seal has also become a persistent problem for small-scale coastal fisheries. Departing from the appraisal theory of emotion, this case study investigated local people’s appraisals of the current situation of small-scale fishery and seals in three fishing villages. Survey results and interviews showed that the fishery-seal situation is perceived as being highly relevant, with negative implications, and local communities and fishers lack the tools to tackle the challenges. Successful management requires not only attention to local context and stakeholder groups, but also to people’s individual interpretation or appraisal of the situation. Place-specific coping strategies should be sought, to balance the seal population with the promotion of small-scale fisheries as a sustainable local industry and as a cultural heritage.