Will a college prep math class hurt my college odds if the rest are APs?

I’m a freshman in highschool, and I currently take honors geometry, and the rest of my classes are the freshman equivalent of an AP (called level 4). Im doing well in all of my level 4 classes (all As and one B+) but in my one honors class, geometry, I’m struggling. My test scores are inconsistent and right now my 80 went down to a 79 because of another bad test, among other good ones. Because of this my parents want me to drop down to college prep (level 2) algebra 2 next year, and while I really don’t want to, I probably will have to either way. With the college prep courses I won’t be able to take typical classes as a junior and senior and it’s way too slow and boring for me, but my grade is not good in honors so I guess I have to just deal with it. So as time comes along my gpa will be good weighted and unweighted, but I think colleges won’t like that I'm taking that class, and frankly i’m kinda embarrassed by it.

I would just like to know, will college prep math classes with AP language, English, history, and science classes get me rejected from top 50 schools?

I still have my final left and if I get an 83 I get my grade back up to a B- and if I get a 97 I get it up to a B. This probably won’t change my parents’ choice tho.