The answer is: Quietly divorce her

Matthew 1:19 - the option of divorce was available under the Law of Moses, Deut 24:1. Normally the circumstances would have to be made public and required witnesses to testify, hence Joseph wanted to do it "privily".

Joseph’s dilemma is a profound reflection on human judgment and compassion. In that era, accusations of infidelity carried hefty social consequences not just for Mary but for Joseph too. His desire to maintain her dignity by opting for a quiet divorce speaks volumes about his character. This isn't just a legal quagmire; it’s a very human story of a man grappling with an emotionally devastating revelation.

Here’s an interesting tidbit: in Hebrew culture, the betrothal period was very serious, often likened to nearly being married, but without the cohabitation. A divorce was necessary to sever that deep bond. In 1st-century Palestine, the act of putting "this matter away" tends to highlight how Joseph suitably juxtaposes public enmity against dignity and discretion.

Moreover, this tale reverberates through countless texts of late antiquity, notably earlier than Jesus in classical literature, where we see similar narratives exploring the tension between personal joyfulness and societal expectations. The crucial response of an angelic messenger (found in Matthew 1:20) redirected Joseph’s path, reminiscent of many biblical figures who received divine direction through dreams, like Jacob and Joseph of the Old Testament.

Understanding these nuances gives us richer insight into not just the world of Jesus but the core values echoing timeless ethical dilemmas—love, mercy, loyalty, and the risks that come with compassion against societal norms. Joseph, a “just man”, indeed becomes a compelling figure in crafting a narrative of acceptance over condemnation.