There are several different ways you can go with regards to Muggelborn characters, and we're thrilled you want to take that route. (Muggleborns rule.)
Some children were taken to orphanages or St Mungos almost immediately (especially if they were children already at Hogwarts), while others had a harder time. We've had characters that were picked up by the Ministry almost instantly, while others had to fend for themselves on the streets. Either way, once they've been taken in by wizarding orphanages they would be told they were wizards and they'd be introduced to the magical world.
Some Muggleborn children that were taken into orphanages were also given the opportunity to live with a Pureblood family. (Though not all the Purebloods were very happy about this). You're welcome to connect with a family if you wanted to go this route. (In fact, I wouldn't mind helping you here, as the Oliveroot's are a pureblood family!)
The other route would be children who were unfortunate enough not to be found. For example, I have a Muggleborn child who was left stranded during the Time Warp with on his twin brother. They hadn't yet displayed any magical ability, and so the Ministry was unable to track them or acknowledge that they were wizards. The result was isolation and them fending for themselves, until some of their powers finally showed.
Though that was how we created these particular characters, you are more than welcomed to say the Ministry was able to track your Muggleborn before showing any magic. You could even say your character suddenly found themselves in their house, but with a different Muggleborn family living there from 1937. It was a hectic and confusing time for everyone within the Wizarding World, so you are free to use that to mold your character as you'd like in that respect.
I hope that answers some of your questions, and if you have anything further I'd be happy to help you out! (You can catch me on Private Chat as well, if you'd like!)