As counter intuitive as it may seem it is easier to manage an all-remote team than it is a hybrid team, where you have some people in the office and some remote:
"Right now, while many companies still have a predominantly remote workforce, often times important conversations are happening on Slack in channels accessible to anyone at the company. But when some people return to the office, a significant portion of those conversations are going to be moved off of Slack to in-person meetings, removing important context for remote workers." - Sean Knight
In essence this communication breakdown happens because you have two communication apps - Slack (or Teams or Workplace etc.) for the remote team and the Office for the in-person team.
Unless you can successfully decommission the office as communication app, in essence creating a single communication channel, you will continue to run into serious issues.