Financial news has been identified as an important alternative information source for modeling market dynamics in recent years. While most of the attention goes to stock markets, the foreign exchange (Forex) market, in contrast, is much less studied. Most of the existing text mining research for the Forex market combine news sentiment with other text features, making the contribution of each factor unclear. To this end, we want to study the role of news sentiment exclusively. In particular, we propose a FinBERT-based model to extract high-frequency news sentiment as a 4-dimensional time series. We examine the efficacy of this news sentiment for Forex market prediction without involving any other semantic feature. Experiments show that our model outperforms alternative sentiment analysis approaches and confirm that news sentiment alone may have predictive power for Forex price movements. The sentiment analysis method seems to have a big potential to improve despite that the current predictive power is still weak. The results deepen our understanding of financial text processing systems.