I always thought polo shirts were a sort of compromise item, the thing you wear when a plain t-shirt feels too lazy and a proper shirt feels like too much effort for a Tuesday. Then I borrowed a knitted one off a friend for a weekend away and realised the collar does something a round neck just can't, it makes a very ordinary outfit look like you dressed on purpose.
Colour wise, I've gone off anything too bright because it ages fast, a polo in a strong colour looks great for one season and then sad and faded the next. Stick to navy, cream, a soft khaki or a proper rust and you'll get years out of it. White is lovely but only if you accept it's a spring and summer colour and you're prepared to actually wash it properly, it shows everything, a coffee drip, a bit of foundation on the collar, all of it.
For pairing, I wear mine with high waisted jeans and loafers more than anything else, tucked in, sleeves left alone rather than rolled, because rolling them on a knitted polo tends to make the cuff lose its shape by lunchtime. Under a blazer it does a surprisingly smart job for something so casual. In cooler months I'll put one under a sleeveless knit or a gilet, which keeps the collar visible and gives you another layer without bulk, genuinely one of the better layering tricks I use once fleece weather starts.
Before buying, check the collar holds its shape with a bit of a press between your fingers rather than flopping straight back down, a soft floppy collar never recovers after a wash. Have a feel of the placket too, the little buttoned bit at the front, because on cheaper versions the buttons sit crooked or the fabric puckers around them, and that's the first thing to go wrong. If it's knitted, stretch the hem gently, a good one springs back, a bad one stays baggy and you'll be tugging it down all day.
Weather wise, a classic cotton polo earns its keep from late spring through to early autumn, breathable enough for warm days without feeling like you've made no effort at all. The knitted version is more of a shoulder season piece, it copes with a chilly morning and a mild afternoon better than cotton ever will, though neither one is doing you any favours in proper rain, the collar just sags.
Next time you're trying one on, check the collar first, press it between two fingers and see if it springs back.