Updated:
Walk down towards the castle and bay, look at the boats, then decide whether lunch or the vineyard paths come next. This is a place where sitting beside the water can be part of the afternoon. You do not have to fit in another town.
The station is above the bay, so leave time and energy for the uphill return. That matters with children or luggage. If you add a boat ride, check the journey home before boarding; returning by train might leave more time ashore. The official walking information shows how the station, castle and vineyards connect.