I have never done a martial art before and I am not that fit, does this matter?

This does not matter at all. In fact, it is good that you have not done a martial art before. This means you will come with an open mind, without bad habits or preconceived ideas. Concerning your physical fitness, this is not important. Phil Allen teaches a wide range of people who vary in age, fitness and experience. He will develop you as an individual, with your unique personality and attributes.