Grid computing in general is a special type of parallel computing. It intends to deliver high-performance computing over distributed platforms for computation and data-intensive applications by making use of a very large amount of resources. The GMRES method is used widely to solve the large sparse linear systems. In this paper, we present an effective parallel hybrid asynchronous method, which combines the typical parallel GMRES method with the Least Square method that needs some eigenvalues obtained from a parallel Arnoldi process. And we apply it on a Grid Computing platform Grid5000. From the numeric results, we will present that this hybrid method has some advantage for some real or complex systems compared to the general method GMRES.