The thing nobody mentions is how a knit tells on itself after a few wears, not a few washes. A cheap one goes thin at the elbows and loses its neckline within a fortnight, sitting lower and wider every time you pull it over your head, while a decent one holds its shape at the collar and cuffs long after you have stopped noticing it is even a jumper and not just something warm you put on.
If you own none of this at all, start plain. A round neck jumper in a mid weight, nothing fluffy, nothing cropped, in navy or a soft grey rather than black, because black knitwear pills visibly and shows every bit of lint from a coat. That one piece will sit under a jacket for work, over a shirt collar for something slightly done, and on its own with jeans on a Saturday without ever looking like you are trying. A cardigan earns its place soon after, ideally one with buttons that actually close properly rather than gaping at the chest, since that is where most cheap cardigans fail first, pulling open right at the point where you need it shut. Knitwear generally comes into its own from October through to March, for the school run, for sitting in a cold office, for a jumper under a coat at a bonfire, though a fine cotton knit earns a place even in a British summer evening when the wind picks up. What separates the good from the disappointing is mostly in the hand feel before you even try it on. Natural fibres, or at least a decent proportion of them, keep their shape and breathe properly, where acrylic goes bobbly and clingy within weeks. Check the ribbing at the cuff and hem, it should have some bounce back when you stretch it, not stay stretched. Check the seams at the shoulder sit where your shoulder actually is, not an inch down your arm, and check buttons on a cardigan are sewn through properly rather than glued or barely anchored, because those are the first thing to go.
On the body, a lot comes down to where it sits at the widest part of you. A jumper that is generous through the body but narrow at the wrist will always look better than one that is fitted everywhere, since knitwear that clings all over tends to show every seam of whatever you are wearing underneath.
Either one, jumper or cardigan, makes an old pair of jeans look like you dressed on purpose rather than by accident.