Should the e-visa be attached in the passport?

No. The evisa should not be attached in the passport. The evisa should be printed by the applicant and presented together with the passport at the border checkpoint of the Republic of Azerbaijan.