Grafana security best practices and hardening

A default Grafana install is built to get you to a dashboard in five minutes. That is the right priority for a first run and the wrong one for anything with a public IP. The gap between the two states is about a dozen settings, most of them one line each. Each one below comes with its reasoning, so you can decide which your setup genuinely needs.

Replace the default admin account

Grafana ships with admin_user = admin and admin_password = admin in conf/defaults.ini, and the first login prompts you to change the password. Changing the password is the minimum. Leaving the username as admin means every credential stuffing script on the internet already has half your login.

On a new instance, create a named administrator through the UI, log in as that account, then demote the original admin user to Viewer or delete it outright once the new account is confirmed working. Two admins existing for the five minutes in between is deliberate. Locking yourself out of Grafana is annoying enough to justify the overlap.

If you prefer to keep one built-in admin, rename it in the config before first start:

[security]
admin_user = grafana-ops
admin_password = $__env{GF_ADMIN_PW}
admin_email = [email protected]

The $__env{} syntax reads the value from an environment variable at load time, which keeps the password out of a file that gets committed by accident. Locked out already? The Grafana default login and password reset guide covers the CLI reset. In Docker the equivalent keys are GF_SECURITY_ADMIN_USER and GF_SECURITY_ADMIN_PASSWORD, or the __FILE variants pointing at a Docker secret.

Keep port 3000 off the public interface

Grafana's built-in HTTP server listens on all interfaces on port 3000 by default, in plain HTTP. Bind it to loopback and put a reverse proxy in front:

[server]
protocol = http
http_addr = 127.0.0.1
http_port = 3000
domain = grafana.example.com
root_url = https://grafana.example.com/
enforce_domain = true

Confirm the bind changed, because an empty http_addr is easy to miss in a file this long:

sudo ss -tlnp | grep ':3000'

You want 127.0.0.1:3000 in that output, not *:3000. Then let nginx or Caddy handle TLS, HTTP/2, certificate renewal and rate limiting. Grafana can terminate TLS itself with protocol = https plus cert_file and cert_key, which is reasonable on a single-purpose box running nothing else, though you then own certificate renewal. Port choices and proxy examples are in the guide to Grafana's default port.

enforce_domain = true redirects requests whose Host header does not match domain. It costs nothing once domain is correct.

What it closes is DNS rebinding, which reads as theoretical until you work through a write-up of one. A page you visit resolves its own domain to a public address, then re-resolves it seconds later to 127.0.0.1, and the browser sends the follow-up request to your loopback service while still treating it as the same origin. Grafana on localhost behind no proxy is a decent target for that. A Host header check ends it.

Cookies and security headers

Once TLS is in front, tell Grafana about it. These live in [security] and every one of them defaults to the permissive value:

[security]
cookie_secure = true
cookie_samesite = lax
strict_transport_security = true
strict_transport_security_max_age_seconds = 31536000
strict_transport_security_subdomains = false
strict_transport_security_preload = false
content_security_policy = true
x_content_type_options = true
x_xss_protection = true
allow_embedding = false

Without cookie_secure = true the session cookie will travel over plain HTTP the moment anything downgrades the connection. strict_transport_security defaults to false with a max age of 86400. Set both lines together, and move to a year once the certificate pipeline has proved itself.

content_security_policy = true switches on the shipped template, which uses a per-request nonce and is written to work with the standard Grafana frontend. Turn it on and leave the template alone. If a plugin breaks under it, add what that plugin needs to content_security_policy_template and keep the header on.

On cookie_samesite, the hardening docs suggest strict. I run lax on instances where people click dashboard links out of chat or an alert email, because strict drops the session on those cross-site navigations and users get a login screen from a link that should have worked. On an instance nobody links into from outside, strict costs nothing.

allow_embedding stays false unless you are deliberately putting panels in an iframe on another site. Setting it true removes the frame-ancestors protection, so if you need embedding, pair it with a CSP template that names the specific parent origin.

Close off self-registration and anonymous access

Both are already off in current Grafana. Homelab tutorials switch them on and the block gets copied wholesale. Verify:

[users]
allow_sign_up = false
allow_org_create = false
auto_assign_org = true
auto_assign_org_role = Viewer

[auth.anonymous]
enabled = false

auto_assign_org_role = Viewer is the safety net for whichever authentication method you add later. When you wire up OAuth or LDAP, new users land in the default organisation with the role this key names, and a value of Editor here means every person in your identity provider can rewrite your dashboards on their first login.

One case justifies turning it on: a read-only status dashboard on a screen in an office, or a public service health page. Grafana's own hardening documentation is direct about the risk, noting that when anonymous access is on, "anyone can make arbitrary queries to any data source" the instance is configured with. Give the anonymous organisation its own datasources, scoped to exactly what the public dashboard needs:

[auth.anonymous]
enabled = true
org_name = Public Status
org_role = Viewer
hide_version = true

hide_version = true stops the version string appearing in the footer and in the /api/health response, which removes the free reconnaissance step of matching your build against a published advisory. That key belongs on any internet-facing instance, anonymous access or not.

Leave brute force protection on

Grafana counts failed logins and blocks after five attempts. The defaults:

[security]
disable_brute_force_login_protection = false
brute_force_login_protection_max_attempts = 5
disable_username_login_protection = false
disable_ip_address_login_protection = true

Username protection is on and IP protection is off out of the box. On an internet-facing instance, flip the IP one. An attacker rotating usernames against a single address is the pattern username-only counting misses:

disable_ip_address_login_protection = false

People disable the whole feature because they locked themselves out and wanted back in faster. Wait it out. Session lifetimes are the companion setting, and the defaults are generous at seven days inactive and thirty days absolute:

[auth]
login_maximum_inactive_lifetime_duration = 2d
login_maximum_lifetime_duration = 14d
token_rotation_interval_minutes = 10

Use service accounts instead of admin credentials

Anything automated that talks to Grafana gets a service account with the narrowest role that works, never a copy of an administrator's password. Provisioning scripts, Terraform, a dashboard-as-code pipeline and your alerting integrations all fall into this category.

Create one under Administration, then Users and access, then Service accounts, assign the role, then add a token. Service account tokens have no expiration date by default, so tick Set expiration date when you create one, and enforce it globally so nobody forgets:

[service_accounts]
token_expiration_day_limit = 90

With that set, Grafana refuses to create a token that expires more than 90 days out. The tokens themselves, the roles and the API calls they enable are covered in the Grafana API and service accounts guide. The practical benefit beyond scoping is revocation: killing one service account breaks one integration, while rotating a shared admin password breaks everything at once and you find out which things used it by reading the pager.

Change secret_key before the instance holds anything

Grafana encrypts the secrets in its database with envelope encryption. Data encryption keys protect datasource credentials and notification settings, and those keys are themselves encrypted with a key encryption key that comes from secret_key in [security]. Of everything on this page, this is the item to do first.

The shipped /etc/grafana/grafana.ini has the line commented out, so the effective value falls back to conf/defaults.ini, where it is a fixed string that is identical in every Grafana installation on earth and readable in the public source tree. Anyone who gets a copy of your grafana.db can decrypt every datasource password in it without doing any work.

openssl rand -hex 32

Put the output in the config, or better, feed it from the environment:

[security]
secret_key = $__env{GF_SECURITY_SECRET_KEY}

Do this on a fresh instance, before any datasource exists. Changing it later leaves already-encrypted secrets unreadable, and the recovery path is a re-encryption run rather than a restart:

sudo grafana cli admin secrets-migration re-encrypt

Back up the database before you run that. I have only ever run it against SQLite, so how long it takes on a Postgres backend with a few thousand encrypted rows, I could not tell you. Do not store the key only in the config file that sits on the same disk as the database it protects, because a single stolen backup then contains both halves.

Stay on a supported Grafana version

Support runs nine months on an ordinary minor and fifteen on the last minor of a major line, with a fresh minor every second month. Leave an instance untouched for a year and it sits outside all of those windows, which is where most of the Grafana incidents that reach the news start. Three minors behind a published advisory, and somebody running a scanner gets there before you do.

The habit matters more than any individual CVE. Subscribe to Grafana's security advisories so a critical fix reaches you the day it lands. Then configure the box to take security updates on its own:

sudo apt-get install -y unattended-upgrades
sudo dpkg-reconfigure -plow unattended-upgrades

Grafana comes from its own APT repository, so unattended-upgrades will not touch it until you add the origin. My preference on a Grafana host is to pin the package and upgrade deliberately, because Grafana runs schema migrations on start and an unattended jump across a minor version is not something I want happening at 03:00.

sudo apt-mark hold grafana
sudo apt-get update && apt-cache policy grafana

Then unhold, upgrade and check the log once a month. Grafana's upgrade guidance lists which versions are still in support, which is the page to read before deciding a version is fine because it works. A homelab can sit on a six month old build without anyone caring. Anything with a public DNS record belongs inside the supported window.

Rate limit the login endpoint with CrowdSec or Fail2ban

Grafana's brute force protection blocks the login, and it does not stop the traffic. On a public instance the attempts keep arriving, filling your logs and costing CPU. Blocking at the firewall is cheaper.

CrowdSec is what I run for this, because the same agent covers SSH and the reverse proxy without a second config language. Installing CrowdSec on Debian covers the agent, and the firewall bouncer with iptables is what turns a decision into a dropped packet. Point it at your nginx access log, since the proxy sees the source IP and Grafana behind loopback does not.

Fail2ban does the same job with a jail matching failed logins in the Grafana log. Either is fine. Running neither on a public Grafana is the version to avoid, since a login form on the open internet gets found within days of the DNS record appearing.

Back up the database and the provisioning directory

Two paths hold everything that would hurt to lose. On a package install with the default SQLite backend:

sudo systemctl stop grafana-server
sudo tar czf /srv/backups/grafana-$(date +%F).tar.gz \
  /var/lib/grafana/grafana.db /etc/grafana
sudo systemctl start grafana-server

On PostgreSQL, pg_dump replaces the database half and the config half stays the same. /etc/grafana carries both grafana.ini and the provisioning directory with your file-defined dashboards, datasources, alert rules and contact points. Store the backup somewhere the Grafana host cannot write to, and test a restore once, because an untested backup is a hypothesis.

Log retention and failed login checks

Grafana's file logger keeps seven days by default, capped at 256MB per file. Seven days is short for a security investigation. Thirty is a better floor on anything internet-facing:

[log.file]
max_days = 30

The lines worth watching name the login path directly:

sudo grep -iE 'invalid username|consecutive incorrect' /var/log/grafana/grafana.log | tail -40

A handful a week is background noise from the internet. A hundred in an hour against one username is someone working through a list, and that is when you go and check that the CrowdSec decision fired. Turning on authn.password:debug in [log] filters gives you per-attempt detail without switching the whole instance to debug, which is one reason to read Grafana's log configuration before you need it.

Trusted Types enforcement on a single instance

One item in the official hardening guide is a judgement call.

Grafana's security hardening documentation describes enabling Trusted Types by adding require-trusted-types-for 'script' to the CSP template, which forces sanitisation of strings passed into DOM injection sinks. It is a real defence against DOM-based XSS. The docs also label it "Currently in development. Trusted types is an experimental Javascript API with limited browser support" and warn that "things may break".

On a Grafana with a handful of trusted users behind TLS, with CSP already on and a nonce-based script policy, the extra coverage is thin and the failure mode is a panel plugin that silently stops rendering. Multi-tenant Grafana, or one where semi-trusted people author panels with HTML content, is a different calculation, and there the move is to turn it on in report mode first and read what it flags. The same reasoning applies to login_cookie_name = __Host-grafana_session: correct, tidy and rarely the thing standing between you and an incident.

Verify what you changed. Grafana's runtime configuration is visible to an admin under Administration, then General, then Settings, which the docs describe as the settings applied to your server via the configuration file and any environment variables. That page is the only reliable answer to "did my setting take", and cookie_secure and http_addr are the two to look at after every edit. Every key on this page is documented in the Grafana configuration reference, and a fresh instance built from the Ubuntu VPS install guide gives you the packaged layout these paths assume.