Create and Connect an Azure Linux VM with an SSH Key Pair

SSH keys are the right way to log in to a Linux VM — no password to guess, brute-force, or leak. This tutorial creates an SSH key pair, provisions an Azure Ubuntu VM that accepts only that key, and connects to it, all from the command line with the Azure CLI. It's the manual counterpart to creating the same VM from C# with Azure.ResourceManager.

How SSH key authentication works

A key pair is two halves: the public key goes on the server (into ~/.ssh/authorized_keys); the private key stays on your machine and never leaves it. The server challenges the client to prove it holds the private half — no secret ever crosses the wire, which is why keys beat passwords.

Create the SSH key pair

ssh-keygen ships with macOS, Linux, Windows, and Azure Cloud Shell:

ssh-keygen \
    -m PEM \
    -t rsa \
    -b 4096 \
    -C "azureuser@geekstore" \
    -f ~/.ssh/geekstore_key

• -t rsa -b 4096 — RSA at 4096 bits. Azure also accepts the newer, shorter Ed25519 keys (-t ed25519, no -b needed) — use those if nothing in your toolchain demands RSA.
• -m PEM — key format Azure expects for RSA.
• -C — a comment to identify the key later.
• -f — output path; you get geekstore_key (private) and geekstore_key.pub (public).

Generating public/private rsa key pair.
Your identification has been saved in /home/user/.ssh/geekstore_key.
Your public key has been saved in /home/user/.ssh/geekstore_key.pub.

The passphrase prompt is worth taking: it encrypts the private key on disk, and ssh-agent remembers it per session so you type it once.

Create the VM with the key

az group create --name geekstore-rg --location centralindia

az vm create \
    --resource-group geekstore-rg \
    --name geekstore-vm \
    --image Ubuntu2404 \
    --size Standard_B2s \
    --admin-username azureuser \
    --ssh-key-values ~/.ssh/geekstore_key.pub \
    --public-ip-sku Standard

Notes on the flags:

• --ssh-key-values uploads your public key; password login is disabled automatically when only a key is supplied.
• If you skip --ssh-key-values, adding --generate-ssh-keys makes the CLI create and use ~/.ssh/id_rsa for you.
• Standard_B2s is a burstable 2-vCPU size — right for dev boxes; check az vm list-sizes --location <region> for options.

The command returns JSON including the publicIpAddress. The portal shows the same VM on its overview page:

You can also paste the public key while creating a VM in the portal — the Administrator account section accepts SSH public key as the authentication type:

Connect

ssh -i ~/.ssh/geekstore_key azureuser@<publicIpAddress>

First connection asks you to confirm the host fingerprint; after that you're in. Save yourself the flags with an entry in ~/.ssh/config:

Host geekstore
    HostName <publicIpAddress>
    User azureuser
    IdentityFile ~/.ssh/geekstore_key

…and connect with just ssh geekstore.

Hardening worth doing immediately

• Lock SSH to your IP instead of the whole internet:

az network nsg rule update \
    --resource-group geekstore-rg \
    --nsg-name geekstore-vmNSG \
    --name default-allow-ssh \
    --source-address-prefixes <your-ip>/32

• Consider Azure Bastion or just-in-time access for production VMs — then the VM needs no public SSH port at all.
• Never copy the private key to the server or into a repo. One machine, one key; add more public keys to authorized_keys for more users.

Copying files and running one-off commands

The same key powers file transfer and remote execution:

# copy a deployment bundle up
scp -i ~/.ssh/geekstore_key ./publish.tar.gz azureuser@<ip>:/tmp/

# run a command without an interactive session
ssh -i ~/.ssh/geekstore_key azureuser@<ip> "sudo systemctl restart geekstore"

With the ~/.ssh/config entry from above these shrink to scp file geekstore:/tmp/ and ssh geekstore "…" — worth setting up on any VM you'll touch twice.

When the connection fails

The three failures that account for nearly every "can't SSH":

• Timeout — the NSG isn't allowing your IP on port 22 (or the VM is deallocated). az vm list -d -o table shows power state and public IP; check the NSG rule's source prefix.
• Permission denied (publickey) — wrong key or wrong user. Confirm -i points at the private key matching the uploaded public key, and the username matches --admin-username. ssh -v shows which keys were offered.
• Host key changed warning — expected if you rebuilt the VM on the same IP; remove the stale entry with ssh-keygen -R <ip> and reconnect.

Azure's serial console (VM → Help → Serial console) gets you a terminal even when SSH is dead — invaluable after a bad firewall change on the VM itself.

Controlling cost between sessions

A stopped-from-inside VM still bills for compute; deallocating releases it:

az vm deallocate --resource-group geekstore-rg --name geekstore-vm   # stop billing compute
az vm start --resource-group geekstore-rg --name geekstore-vm        # resume later

For a dev box you use on workdays, auto-shutdown covers the forgetting:

az vm auto-shutdown --resource-group geekstore-rg --name geekstore-vm --time 1900

Note the public IP can change across deallocate/start unless the IP was created --sku Standard static (ours was) — one reason the tutorial pinned it.

Clean up

az group delete --name geekstore-rg --yes --no-wait

Deletes the VM, disk, IP, and network in one stroke — a B2s VM left running costs real money.