Hi VC,
If your only real motivation is sport, then I suggest you try to find something that will keep you close to sport. Isaac makes a good point& don't rush into anything. I am assuming you are in year 12? With your maths and science subjects you should be able to get into "most" degrees (if you do choose uni).
I know where you are coming from, as I had a similar situation to you when I was in high school. Only motivation was sport, to cut it short I did sports science and am now doing a MSc. S&C.
Out of the possibilities you have mentioned some points to consider (keep in mind this is only my opinion, feel free to disagree):
Physio: you will be working with the injured, all (or most of) the time. A lot of physios work with elderly, this is where the bulk of "most" physios bankroll will be, however if you choose right you can work with athletes, suss out different courses to keep you close to sport. As someone mentioned there are only so many spots for sports physios.
Sports administrator: essentially you are an administrator, I never really considered this so may get advice from someone in the field rather than me, but really that's it, your "hands on" with sport will be less than the other jobs you mentioned I would assume (could be wrong, don't discount it).
Sports scientist: if you are looking to get involved I strongly suggest work experience with an institution you wish to work for during your uni, generally in 2nd or 3rd year you can do this as a subject called a "field study" (or similar), such as the Vic inst of sport or similar. In this field you can be responsible for a whole range of jobs, you may specialise as a physiologist, biomechanist, psychologist (yuck), anthropometrist etc etc. You will probably be involved with athletes, positions will be limited the higher the level of the athlete.
Some one mentioned working as a personal trainer, you can do this without going to uni, all you need is a tafe degree. However if you want to work with more advanced athletes I would go to uni. I have done cert IV in fitness (personal training) during my uni degree (someone said it was a good idea for getting a job&true& if you want to work with general population in a gym), but I liken it to year 3 maths versus differential calculus when comparing to a uni degree (again have done both myself and passed both so I feel I can comment).
I hope this helps, as I said I may be biased towards the sports science field as it is the path I opted for. Without bagging you, honestly I think you should do some research yourself, you have made a start by posting here, but I mean go in "the field" and have a look, work experience can be very valuable, whether you hate it or love it, it will tell you the answer (shit posting on this site I nearly spelt that awnser& too far, sorry).
Good luck VC, I wish you all the best mate, and hope you find what you are looking for.