Do colleges take account in Phys Ed grades?

Colleges will look at your transcript and your overall GPA, however, if you have a 97 GPA I don't think a few B's for gym will matter. It also depends on the type of school you're applying to.

With a 97 GPA you shouldn't really have much to worry about and your chances are definitely not ruined. Colleges will look at your transcript (of course) but different schools will look at them in different ways. Some may look only at core classes, some may look at every class, some might drop your 9th grade classes. Keep in mind your high school will also send a high school "profile" to any school you apply to. This profile has lots of demographic information and typically also provides a breakdown of how grades will work. The school will use this additional information when considering your profile as well. Sometimes a chart might be included in the profile showing the GPA ranges of the previous year seniors - I would imagine if your school does this you would still be near the top depending on your school.

That was a long way of saying you don't need to worry!

Bad grades always hurt, especially if they're not in honors or ap classes. that being said, many top tier colleges will recalculate your gpa with only core classes, and use that.

Most of the time PE isn't considered in your GPA (like the UC system) so you should be fine.