We have proposed a robust, secure and efficient image encryption algorithm based on chaotic maps and algebraic structure. Nowadays, the chaotic cryptosystems gained more attention due to their efficiency, the assurance of robustness and high sensitivity corresponding to initial conditions. In literature, there are many encryption algorithms that can simply guarantees security while the schemes based on chaotic systems only promises the uncertainty, both of them can not encounter the needs of current scenario. To tackle this issue, this article proposed an image encryption algorithm based on Lorenz chaotic system and primitive irreducible polynomial substitution box. First, we have proposed 16 different S-boxes based on projective general linear group and 16 primitive irreducible polynomials of Galois field of order 256, and then utilized these S-boxes with combination of chaotic map in image encryption scheme. Three chaotic sequences can be produced by the disturbed of Lorenz chaotic system corresponding to variables x, y and z. We have constructed a new pseudo random chaotic sequence k i based on x, y and z. The plain image is encrypted by the use of chaotic sequence k i and XOR operation to get a ciphered image. To show the strength of presented image encryption, some renowned analyses are performed.