All TLS/HTTPS clients have a set of Certificate Authority keys which they trust. Your client will only accept a public key which is signed by a trusted CA’s key. A proper CA will not sign a key for a domain when it has not verified that the entity that wants it’s key signed actually controls the domain.

So there are two things that certificates are for. You already understand the first one, which is the cryptography itself. You can use them to encrypt your traffic so that information sent over the connection is not in plain text.

The second thing certificates do, is the answer to your dilemma. Identification. For your browser to trust a website’s certificate, the certificate has to be valid for that website. What makes a certificate valid? The certificate has to have been signed by a trusted Certificate Authority, and the name on the certificate must match the website you’re visiting. If you were to ask “What makes a certificate authority trusted?” The answer is that your web browser and/or operating system come preloaded with certificates for trusted certificate authorities. These special certificates were used to sign the certificate of the website you’re visiting, which is another thing your browser checks for. A malicious third party can’t (easily) obtain a valid certificate/key pair for a domain that they don’t own. If your browser was presented with a fake certificate from the malicious third party, it would not connect and would warn you that your connection isn’t secure and would explain why.

Now if more specifically, you’re wondering that if a malicious third party takes any given website’s public certificate, can it use that to decrypt your session? After all, that public cert is signed and trusted. The answer to that, is that when a certificate is created, so too is a private key file created. This private key is never presented to the public, and it’s the only thing that can decrypt sessions that were encrypted by its paired public certificate. So that third party could install that certificate on a web server theoretically, but they wouldn’t actually be able to decrypt anything because they don’t have the private key for the legitimate certificate.

So in order for a man in the middle attack like this to work, they’d have to obtain not only a legit websites public certificate, but also the corresponding private key. OR, the third party would need to get access to your PC, and install its own certificate authority signing cert, so that it’s fake, self signed certificates are trusted by your browser. Both of these are possible, but at that point you’re not talking about an unknown man in the middle, the man would have to compromise one of the two ends.