Allah is a transcendental entity and possesses all excellences. At the same time, creation is a means of grace and Allah is graceful. His unlimited grace calls for him to create whatever has the potential to be created. Therefore, Allah created because he was graceful, meaning that the reason behind creation lies in his grace and mercy. Moreover, knowing that Allah’s attributes of essence are in oneness with his entity, one can say that the reason behind creation is Allah himself.
Allah created man with two inner calls to good and bad, as well as two outside callers to good and bad, the prophets and Satan, resulting in the potential to reach the highest levels of perfection and also declining to the lowest levels of human value. If man, despite his carnal and satanic desires, takes the path of truth and perfection, then he will be greater than the angels, because they do not have carnal and satanic desires. And if he chooses the path of evil, he will be worse than animals, because they do not have a spiritual aspect like humans do.
If Allah created man with all excellences he could possibly possess, this perfection had not been reached through his free will (making the creation of man in vain). Therefore, the purpose behind man’s creation is only met when he is created with the potential of perfection and when he reaches it through his free will. However, the fact that unbelievers and sinners do not fulfill the true purpose behind creation does not contradict Allah’s reason for creation, because Allah wanted them to have the chance to choose the right or wrong path and reach perfection if they chose to. If Allah made it impossible for man to take the path of evil, then choosing the path of perfection and development would not be through free will either, (and this would contradict the wisdom behind creation).
In order to explain the answer, we must shed light on a few points:
A: The wisdom behind creation:
1- Due to the fact that Allah is Wajibul Wujud (Necessary being) and His existence is not dependant on anything else, he has no limits, is transcendental and possesses all perfection.
2- One of His excellences and virtues is his unlimited grace and mercy. The Almighty says in the Quran:
“And the bounty of your Lord is not confined” 
There are no limits to Allah’s grace, therefore when he does not grant something, it is because of the limitations of the receiver not the granter, and what is possible to be granted, will be granted.
3- All perfection stems from Wujud (existence) and all limitations and deficiencies stem from the lack of Wujud (existence). For example knowledge is a type of perfection and ignorance (which is a lack of knowledge) is a limitation and deficiency. Another example would be power, the opposite of disability, which is a type of perfection. Therefore, Wujud (existence) is the source of all goodness and blessing, meaning that all limitations are a form of nonexistence and lack of Wujud (existence).
4- Taking the third point into consideration, one can conclude that Allah’s mercy and grace will only be fulfilled through creation. Therefore, the inevitable result of being graceful is to create.
In other words, if something has the potential to be created and Allah does not create it, this would mean preventing goodness because Wujud is the source of goodness, and therefore such an act would be considered stinginess, and stinginess cannot be attributed to Allah. In the light of all this, we conclude that if we were asked “what caused Allah to create?” we would reply: His grace caused him to create.
5- Allah’s attributes are in complete oneness with his entity. The attributes of human beings and other objects are not in oneness with their essence. For example, an apple has an essence and also has certain attributes such as being red or sweet. Being red or sweet has no unity with the essence of the apple. An apple can be sour and green, but its essence will still exist.
The oneness of Allah’s essence and his attributes is a very deep issue that can be studied in the science of theology. However, what is important to know is that being graceful, which is the “final cause” or “purpose” (Ellate Ghayi) of creation, is Allah’s entity and not something other than his essence. Therefore if we were asked: “Why did Allah create creatures?” We will answer: Because He is Allah. So the final cause is actually Allah himself. This is exactly what philosophers mean when they say: The final cause and “efficient cause” are the same when it comes to Allah’s acts.  Moreover, we might be able to find traces of this meaning in some Quranic verses  like:
All matters return to him. 
B: What was mentioned explained the creator’s wisdom behind the whole concept of creation, but to explain the wisdom behind the creation of a certain being like man requires the explanation of another point and that is the particular perfection Allah wanted to create through creating man.
Explanation: The inevitable result of Allah’s grace is for Him to create any excellence that has the potential to be created. Before the creation of man, He had already created beings like the angels that possessed all potential excellences from the beginning, meaning that they possessed all their perfection in actuality. Therefore, they would never reach a new form of perfection and their existential perfection could never develop to a higher level. Allah quotes the angles in the Quran saying:
“There is none among us but has a known place. Indeed it is we who are the ranged ones. Indeed it is we who are those who celebrate Allah’s glory.” 
Imam Ali stated:
“Then He created the openings between high skies and filled them with all classes of His angels. Some of them are in prostration and do not kneel up. Others in kneeling position and do not stand up. Some of them are in array and do not leave their position. Others are extolling Allah and do not get tired!” 
They worship Allah and do not get tired and this is a perfection Allah has granted them, and they cannot choose to disobey Allah. Allah says in the Quran:
“They do not venture to speak ahead of Him, and they act by His command.”  And He also says: “Over that fire are [assigned] angels, severe and mighty, who do not disobey whatever Allah has commanded them, and carry out what they are commanded.” 
Due to His unlimited grace, Allah wanted to create a higher perfection in addition to the perfection that angels possessed, and that is the perfection of man reaching excellence through his free will; meaning that He created a creature that could decide and strive to possess all perfections through his free will. Therefore, He created man, a being that does not possess all of his potential excellences from the beginning, but has the power to reach all of them as well. It is very clear that the perfection man reaches is of a higher level compared to the excellence angels have been granted. Imam Ali states:
“Allah created the angels from the ‘Mind’ and did not give them desire and created animals from desire and did not grant them intelligence, and he created man from mind and desire. Whoever that his mind prevails over his desire will be higher than the angles and whoever that his desire prevails over his mind will be lesser than animals”. 
Therefore all causes of creation are the grace of Allah. The inevitable result of Allah’s grace is for Him to create this possible excellence, an excellence higher than any other.
C: Why didn’t Allah create man perfect in the first place?
In light of what was said, one can conclude that the purpose of creation will only be fulfilled when man is created with the potential to reach perfection and does so through his free will. If he possessed that perfection from the beginning, such a thing would definitely not be considered a perfection he earned on his own and the purpose of his creation would not be fulfilled.
We must keep in mind that even reaching the smallest level of perfection is considered possessing excellence through free will and, therefore, fulfills the purpose of creation to that extent.
D: Sinners and Unbelievers:
Even if someone does not reach the lowest level of perfection and his life is full of disbelief and disobeying Allah, his acts still do not contradict the wisdom behind creation, because he too has actualized his potentials. Man has the potential to decline to the lowest levels of value, and Allah has created him in a way that he can choose the way of perfection and truth or the way of evil. Therefore, even an unbeliever has not acted against the takwini (having to do with creation) will of Allah. Obviously, Allah wants man to move towards perfection and does not wish him to go astray. In other words, in creating man Allah had two wills, the first was takwini and the second was tashri’i: his takwini will is for all humans to have the chance to actualize their potentials while his tashri’ri will is for humans to take the path of perfection and salvation.
Taking what was said into consideration, one can say that a believer has acted both according to Allah’s tashri’i and takwini will, but an unbeliever or sinner has only acted according to Allah’s takwini will and has not followed Allah’s tashri’i will.
Note: Considering the importance of this issue and the many narrative reasons, there is room for more research and questioning on your part.
 See: Tabatabai, Muhammad Husein, Al-Mizan, vol. 8, p. 44; Misbah Yazdi, Muhammad Taqi, Ma’aarefe Quran, vol. 1, p. 154.
 Baqarah:210; Aal Imran:109; Anfal:44; Hajj:76; Fater:4; Hadid:5.
 Saffat:164 and 166.
 Nahjul-Balaghah, sermon 1.
 Wasa’il al-Shia, vol. 11, p. 164.