Autonomous agents are widely applied to automate interactions in robotics, e.g. for selling and purchasing goods on eBay, and in financial markets, e.g. in the form of quote machines and algorithmic traders. Current research investigates efficient economic mechanisms that fully automate the provisioning and usage processes of Grid-based services. On the one hand, consumers want to allocate resources on demand for their various applications, e.g. data sharing, stream processing, email, business applications and simulations. On the other hand, providers of Grid services want to govern business policies to meet their utilization and profit goals. The above-mentioned processes are not manually manageable, however, because decisions need to be taken within milliseconds. Therefore, such processes have to be automated to minimize human interactions. Hence, market mechanisms and strategic behavior play important roles when it comes to achieving automated and efficient allocation of Grid services. The paper begins by presenting a framework for automated bidding, providing a methodology for the design and implementation of configurable bidding strategies. Second, it presents a novel bidding strategy based on a reinforcement learning technique. This strategy is designed to automate the bid generation processes of consumers and providers in various market mechanisms. Third, the behavior and convergence of the strategy is evaluated in a centralized Continuous Double Auction and a decentralized on-line machine scheduling mechanism against selected benchmark bidding strategies. Fourth, we define a bidding language for communicating consumer and provider preferences to the market as well as report back the match of the market-based allocation process.