The on-chip caches usually consume a significant amount of energy in modern microprocessors. This paper presents an I/D filter scheme to reduce the energy consumption of united L2 caches shared by instructions and data. By adding an I/D indicator bit, the cache block is classified into I-block and D-block. For instruction and data accesses, only the corresponding blocks instead of all the blocks in the same set selected are accessed. By this method, we can easily filter the unnecessary way activities and save the energy consumption. This technique uses a small amount of additional hardware without increasing the cache access latency, and the area overhead is negligible. Simplescalar simulator and CACTI were used to evaluate the performance of our proposed architecture, the results shows that the I/D filter scheme is energy efficient for set-associative caches.