Words have literal meanings and technical \ specialized meanings. For example:
Zakat literally means to increase or purify. However when used in Islamic texts and discussions, without any further context or qualifications, it is understood to mean the purification of wealth through giving of charity - not any other type of purification.
Sawm literally means abstinence, however as a technical term in Islam it means the ritual of fasting i.e. to abstain from food, drink and sexual pleasures from dawn till sunset.
Salah literally means supplication, however as a technical term in Islam it means a worship ritual involving certain movements and recitations.
Hajj literally means to intend towards something great. However as a technical term in Islam it means the rituals of pilgrimage that are performed annually in a specific month towards the sacred Masjid in Makkah.
In the same way to do Jihad literally means 'to struggle', and is occasionally used to mean different types of struggle. However when used as a standalone term without any more context or qualification, it is understood to be in its technical meaning, which is defined as: fighting against the disbelievers. And at times for preparation of fighting such as the financing of weapons, transport and supplies etc. (which is Jihad with one's wealth) or motivating.
قال: وما الجهاد؟ قال: أن تقاتل الكفار إذا لقيتهم ، قال: فأي الجهاد أفضل؟ قال: من عقر جواده وأهريق دمه
A man asked: "What is Jihad?"
The Prophet ﷺ said: "To fight the disbelievers when you meet them."
The man asked: "Which Jihad is most superior?"
The Prophet ﷺ said:"That in which his steed is hamstrung and his blood is spilled."
— Musnad Ahmad
أن الجهاد في سبيل الله إذا أطلق فلا يقيم بإطلاقه إلا على مجاهدة الكفار بالسيف حتى يدخلوا في الإسلام أو يعطوا الجزية عن يد وهم صاغرون
If Jihad in the way of Allah is mentioned categorically then it does not mean anything except struggling against the disbelievers with the sword until they accept Islam or pay Jizyah while being subdued.
— Muqaddimmat al-Mumahhidaat
One can also look at ahadith where fighting is compared with other actions. Note that prayer, fasting, seclusion, migration and pilgrimage are also types of spiritual and physical struggle but in these ahadith Jihad is used specifically as a name for fighting as distinct from the other types of struggle:
جاء رجل إلى رسول الله صلى الله عليه وسلم فقال دلني على عمل يعدل الجهاد قال لا أجده ـ قال ـ هل تستطيع إذا خرج المجاهد أن تدخل مسجدك فتقوم ولا تفتر وتصوم ولا تفطر قال ومن يستطيع ذلك قال أبو هريرة إن فرس المجاهد ليستن في طوله فيكتب له حسنات
A man came to Allah's Messenger (ﷺ) and said, "Instruct me as to such a deed as equals Jihad (in reward)."
He replied, "I do not find such a deed." Then he added, "Can you, while the Muslim fighter is in the battle-field, enter your mosque to perform prayers without cease and fast and never break your fast?" The man said, "But who can do that?" Abu- Huraira added, "The Mujahid (i.e. Muslim fighter) is rewarded even for the footsteps of his horse while it wanders bout (for grazing) tied in a long rope."
— Bukhari
أن رجلا، أتى النبي صلى الله عليه وسلم فقال أى الناس أفضل فقال رجل يجاهد في سبيل الله بماله ونفسه قال ثم من قال مؤمن في شعب من الشعاب يعبد الله ربه ويدع الناس من شره
A man came to the Prophet (may peace he upon him) and said:
Who is the best of men?
He replied: A man who does Jihad (fights) in the way of Allah spending his wealth and staking his life.
The man then asked: Who is next to him (in excellence)?
He said: Next to him is a believer who lives in a mountain gorge worshipping his Lord and sparing men from his mischief.
— Muslim
أن رسول الله صلى الله عليه وسلم سئل أى العمل أفضل فقال إيمان بالله ورسوله قيل ثم ماذا قال الجهاد في سبيل الله قيل ثم ماذا قال حج مبرور
Allah's Messenger (ﷺ) was asked, "What is the best deed?" He replied, "To believe in Allah and His Apostle (Muhammad).
The questioner then asked, "What is the next (in goodness)? He replied, "To participate in Jihad (religious fighting) in Allah's Cause."
The questioner again asked, "What is the next (in goodness)?" He replied, "To perform Hajj 'Mubrur (pilgrimage)"
— Bukhari
قال رسول الله صلى الله عليه وسلم لا هجرة بعد الفتح ولكن جهاد ونية، وإذا استنفرتم فانفروا
Allah's Messenger (ﷺ) said, "There is no Hijra (i.e. migration) after the Conquest (of Mecca), but Jihad and good intention remain; and if you are called (by the Muslim ruler) for fighting, go forth immediately.
— Bukhari
One can also see the definitions of Jihad which is done by scholars in manuals of jurisprudence:
الجهاد هو الدعاء إلى الدين الحق، والقتال مع من امتنع عن القبول بالنفس، والمال
Jihad is inviting the people to accept the true religion and fighting against those who refuse to accept it, by one's life and wealth
— Al-Bahr ar-Raiq and Fatawa 'Alamgiri (Hanafi)
الجهاد: قتال مسلم كافراً غير ذي عهد؛ لإعلاء كلمة الله أو حضوره له، أو دخول أرضه له
Jihad is fighting by a Muslim against a disbeliever who does not have a treaty for the sake of superiority of the word of Allah, or being present in the battle or entering the land of the disbelievers
— al-Fawaakih ad-Dawaani (Maliki)
ولأن الجهاد هو القتال
Jihad is fighting
— al-Muhadhdhab fi Fiqh al-Imam ash-Shaafi‘i also see إتحاف الأريب (Shafi'i)
تعريف الجهاد ... شرعا: عبارة عن قتال الكفار خاصة
Definition of Jihad: ... In Shariah: it means fighting the disbelievers specifically
— al-Mubdi‘ fi Sharh al-Muqni‘ (Hanbali)
So if an orthodox Muslim or scholar talks about Jihad, then they typically mean this type of Jihad. And this is the 'real' meaning and connotation of the word, and the first reaction to it - unless there is implicit context or explicit evidence within the conversation to indicate that they are talking about something else.