Mohammad Ali was born in 1878 in Rampur, India.[1] His father, Abdul Ali Khan, died when he was five years old.[3] He was the brother of Shaukat Ali (politician) and Zulfiqar Ali. "His mother Abadi Begum, affectionately known as Bi Amma, inspired her sons to take up the mantle of the struggle for freedom from Colonial rule. To this end, she was adamant that her sons were properly educated."[3]

Despite the early death of his father, Jouhar attended the Darul Uloom Deoband, Aligarh Muslim University and, in 1898, Lincoln College, Oxford, studying modern history.[5]