AI is critically analyzed through the lens of the black box metaphor, a useful methodological tool within the field of science and technology studies. The main purpose is to show how the black box of AI is constructed, to present the nuances related to it, how to unpack it, and to present the full extent to which one can use the black box metaphor to reflect on AI. By trying to answer basic research questions about AI's black box, its construction through a co-production of technoscientific and social factors is revealed along with ways to pry open its black box. Understanding the construction of the black box is the first step to start opening it. Its implications for society lead to the mandate for transparency to mitigate the adverse effects stemming from blackboxing procedures. Contingent on the black box of AI and the transparency mandate is trust in AI. Increasing trust in AI presents a difficult but necessary task as AI is, ultimately, a technosocial phenomenon that both shapes and is shaped by society. Finally, the relationship between the biological and the mechanical domain by means of the black box metaphor and AI research is evaluated.