This paper introduces the detailed process of applying reinforcement learning to solve market participant bidding strategy problem. The process includes the setup of market clearing environment, reinforcement learning structure, and Q-learning algorithm. A comprehensive study on three specially designed problems demonstrates the Q-learning method can achieve significantly higher profit than the baseline method, which employs marginal cost as the offer price. The study provides insights to the learning process and the performance of Q-learning and demonstrates the performance varies with the changing condition of the environment, and tends to degrade with more complex patterns or random disturbances in the environment.