.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
.svg)
I once spent twenty minutes tailing /var/log/grafana/grafana.log inside a container that had never written a single byte to it. The file was there, it was zero bytes, and everything I wanted was sitting in docker logs the whole time.
The log path is not decided by one setting in one file. It comes out of a five-step configuration load order, and the packaging (deb, rpm, Docker, tarball) injects values partway through that order. Once you can see the order, moving the log file becomes a two-line change.
Default log locations by install method
Grafana's built-in default for [paths] logs is data/log, relative to the working directory. Almost no running instance uses it, because every packaged install overrides it before Grafana finishes starting.
| Install method | Log destination |
|---|---|
| deb / rpm package (apt, yum, dnf) | /var/log/grafana/grafana.log |
| Binary tarball or manual install | <install dir>/data/log/grafana.log |
| Homebrew on macOS | /usr/local/var/log/grafana/grafana.log |
| Docker container | stdout only, read with docker logs |
The Grafana troubleshooting documentation puts it as "Usually located at /var/log/grafana/grafana.log on Unix systems or in <grafana_install_dir>/data/log on other platforms and manual installations". That is accurate. It does not say where /var/log/grafana comes from, which is the next section.
Where /var/log/grafana comes from on a package install
It is not in /etc/grafana/grafana.ini. The shipped config file has the whole [paths] section commented out with leading semicolons, so if you go looking for the string /var/log/grafana in there you will not find it.
It lives in the environment file that the systemd unit reads:
cat /etc/default/grafana-serverOn Debian and Ubuntu that file sets the paths the service will use:
GRAFANA_USER=grafana
GRAFANA_GROUP=grafana
GRAFANA_HOME=/usr/share/grafana
LOG_DIR=/var/log/grafana
DATA_DIR=/var/lib/grafana
MAX_OPEN_FILES=10000
CONF_DIR=/etc/grafana
CONF_FILE=/etc/grafana/grafana.ini
RESTART_ON_UPGRADE=true
PLUGINS_DIR=/var/lib/grafana/plugins
PROVISIONING_CFG_DIR=/etc/grafana/provisioning
PID_FILE_DIR=/run/grafanaThe same postinstall script that creates those directories also locks down the config: everything under /etc/grafana ends up owned root:grafana with files at mode 640 and directories at 755. So the grafana user can read grafana.ini and nobody else on the box can, which matters the moment you put a real SMTP password or a datasource credential in there.
Those variables get expanded into the unit's ExecStart line:
ExecStart=/usr/share/grafana/bin/grafana server \
--config=${CONF_FILE} \
--pidfile=${PID_FILE_DIR}/grafana-server.pid \
--packaging=deb \
cfg:default.paths.logs=${LOG_DIR} \
cfg:default.paths.data=${DATA_DIR} \
cfg:default.paths.plugins=${PLUGINS_DIR} \
cfg:default.paths.provisioning=${PROVISIONING_CFG_DIR} \
cfg:default.paths.bundled_plugins=${DATA_DIR}/plugins-bundledNote the cfg:default. prefix on every one of those arguments. It drops the packaging's values into the defaults layer, underneath anything you write in grafana.ini, and that one detail is why editing grafana.ini works at all. The binary is grafana server, two words. The unit is still named grafana-server.service, so systemctl restart grafana-server stays correct even though grafana-server as a command now prints a deprecation warning.
Grafana configuration precedence, in load order
Grafana builds its configuration in five passes. Each pass overwrites whatever the previous one set:
conf/defaults.ini, shipped with the package. The docs are blunt about this one: "Don't change this file."- Command line arguments of the form
cfg:default.<section>.<key>=<value>. These write into the defaults layer, as if they had been in defaults.ini. - The custom config file:
/etc/grafana/grafana.inion packages,conf/custom.inion a tarball. - Environment variables of the form
GF_<SECTION>_<KEY>, uppercased, with dots and hyphens swapped for_, which gives youGF_LOG_LEVEL,GF_LOG_MODEandGF_PATHS_LOGS. - Command line arguments of the form
cfg:<section>.<key>=<value>, without thedefault.part. These sit at the top of the chain and nothing overrides them.
Step 2, the defaults layer, is where the packaged systemd unit writes, and the Docker entrypoint writes there too. Your grafana.ini is step 3 and your GF_* environment variables are step 4. Both sit above the packaging, so both win.
Move the Grafana log directory
Using grafana.ini
Because the config file loads after the cfg:default.* arguments, setting the path in /etc/grafana/grafana.ini genuinely relocates the log, with the unit still passing cfg:default.paths.logs=/var/log/grafana on every start:
[paths]
logs = /srv/grafana-logsCreate the directory and hand it to the service account first, or Grafana will fail to open its log file and you will be back to reading journald:
sudo mkdir -p /srv/grafana-logs
sudo chown grafana:grafana /srv/grafana-logs
sudo chmod 750 /srv/grafana-logs
sudo systemctl restart grafana-serverThe packaged unit ships with ProtectHome=true and ProtectSystem=full. A log directory under /home or somewhere in /etc will be invisible or read-only to the service no matter how the permissions look from your shell. Pick a path under /srv or /var, or add a drop-in with ReadWritePaths= for the directory you want.
Using /etc/default/grafana-server
On a deb or rpm install this is the route I take. Change one line:
sudo sed -i 's#^LOG_DIR=.*#LOG_DIR=/srv/grafana-logs#' /etc/default/grafana-server
sudo systemctl restart grafana-serverIt is tidier because LOG_DIR is the value the rest of the packaging already agrees on. Either way, the postinstall script hardcodes /var/log/grafana and /var/lib/grafana, creating them, chowning both to grafana:grafana and setting mode 755 on every upgrade. A relocated log directory sits outside that, so its ownership stays yours to keep correct.
Whichever route you pick, confirm it landed rather than trusting the config:
sudo ls -la /srv/grafana-logs/
sudo journalctl -u grafana-server --since "2 minutes ago" | tail -20The [log] section and what each key does
Here is the section with its shipped defaults, taken from conf/defaults.ini:
[log]
mode = console file
level = info
filters =
user_facing_default_error = "please inspect Grafana server log for details"mode is space separated and accepts console, file and syslog in any combination. level takes debug, info, warn or error, and it applies globally unless a filter overrides it for one logger. The user_facing_default_error string is what users see in the UI in place of a backend error, which is why so many Grafana screenshots on forums say "please inspect Grafana server log for details" and nothing else.
[log.console] and [log.file]
Each mode gets its own subsection, and each one can carry its own level and format. Leaving level blank inherits the global one.
[log.console]
level =
format = console
[log.file]
level =
format = text
log_rotate = true
max_lines = 1000000
max_size_shift = 28
daily_rotate = true
max_days = 7format accepts text, console or json. The console format is coloured when stdout is a terminal and falls back to plain text otherwise, so in a container or under systemd you get the same logfmt output either way.
max_size_shift = 28 looks like a typo. It is a bit shift: 1 << 28 bytes, which is 256MB per file. With daily_rotate = true and max_days = 7 you get a week of history, capped at a million lines or 256MB per file, whichever comes first. That is plenty for a single Grafana instance.
[log.syslog]
[log.syslog]
level =
format = text
network =
address =
facility =
tag =network takes udp, tcp or unix, and leaving it blank uses the local unix socket. facility accepts user, daemon and local0 through local7. If you are already collecting syslog centrally this is the least effort way to get Grafana into that pipeline, though on a systemd box the journal is already capturing console output for free.
Per-logger levels with filters
filters raises the level for one named logger and leaves everything else at info. Global level = debug on a busy Grafana produces an unreadable firehose: every HTTP request, every datasource query, every cache lookup and every token rotation. A filter gets you the ten lines you were after.
The syntax is logger:level, comma separated:
[log]
level = info
filters = sqlstore:debugGrafana names its loggers in the code, and the names show up in every log line as logger=, so the fastest way to find the one you need is to grep your existing log for the component you care about. A few that come up often, all verified against current source:
| Logger | Covers |
|---|---|
sqlstore | Database access, migrations, connection setup |
sqlstore.xorm | The generated SQL itself |
plugin.loader | Plugin discovery and load failures |
provisioning | Provisioned dashboards, datasources, alert rules and contact points |
datasources | Datasource service operations |
notifications | SMTP and email sending |
rendering | Image renderer plugin |
http.server | The HTTP server itself |
ngalert | Unified alerting |
authn.password | Username and password login attempts |
A few combinations worth keeping in your notes. Debugging why a provisioned dashboard is not appearing:
[log]
filters = provisioning:debug,plugin.loader:debugChasing an email that never arrived, which pairs well with fixing the Grafana SMTP not configured error:
[log]
filters = notifications:debug,ngalert:debugAnd when the UI is slow and you suspect the database rather than the browser:
[log]
filters = sqlstore:debug,sqlstore.xorm:debugThat last one gets loud fast. Turn it on, reproduce the slow page once, turn it back off. I have never regretted leaving a filter on for ten minutes and I have definitely regretted leaving one on for a week.
Why grafana.log is empty in a Docker container
The official image bakes in GF_PATHS_LOGS=/var/log/grafana, so the directory exists and looks like the place logs should be. The entrypoint script then ends with this:
exec grafana server \
--homepath="$GF_PATHS_HOME" \
--config="$GF_PATHS_CONFIG" \
--packaging=docker \
"$@" \
cfg:default.log.mode="console" \
cfg:default.paths.data="$GF_PATHS_DATA" \
cfg:default.paths.logs="$GF_PATHS_LOGS" \
cfg:default.paths.plugins="$GF_PATHS_PLUGINS" \
cfg:default.paths.provisioning="$GF_PATHS_PROVISIONING"There it is: cfg:default.log.mode="console". The image drops file from the default mode so container logs go to stdout and into your runtime's logging driver. The file never gets created because file mode is off, and the directory sits there empty looking broken.
Because that argument goes into the defaults layer, an environment variable overrides it. Turn file mode back on and give the directory somewhere to live:
docker run -d \
--name=grafana \
-p 3000:3000 \
-e "GF_LOG_MODE=console file" \
-v grafana-storage:/var/lib/grafana \
-v grafana-logs:/var/log/grafana \
grafana/grafanaNow docker exec grafana ls -la /var/log/grafana shows a real grafana.log, and docker logs still works because console is still in the mode list. The same thing in a Grafana Docker Compose stack is one line under environment:. A mounted grafana.ini with [log] mode = console file does the job too, for the same reason: the config file loads after the defaults layer.
Bind mounts carry one extra failure here. The container runs as uid 472 with gid 0, so a host directory owned by your login user will not be writable and Grafana exits at startup complaining it cannot open the log file. A named volume avoids it. Why the image settled on gid 0 rather than a grafana group, I do not know.
Reading Grafana logs in practice
On a systemd host, journald has everything console mode produced, with or without file mode on:
sudo journalctl -u grafana-server -f
sudo journalctl -u grafana-server --since "1 hour ago" -p errThe file, when you have one:
sudo tail -f /var/log/grafana/grafana.log
sudo grep -i 'level=error' /var/log/grafana/grafana.log | tail -50And in a container:
docker logs -f grafana
docker logs --since 10m grafana 2>&1 | grep -i errorThe startup banner tells you which config won
Every Grafana start logs the outcome of the precedence chain before it does anything else, and reading those eight lines answers most "why is my setting being ignored" questions without opening a single config file:
sudo journalctl -u grafana-server -n 200 --no-pager | grep -E 'Config loaded|Config overridden|^.*Path 'You get Config loaded from with the file it used, then a Config overridden from command line line for every cfg: argument the unit passed, then a Config overridden from Environment variable line for every GF_* variable Grafana picked up, with secrets redacted. After those come Path Home, Path Data, Path Logs, Path Plugins and Path Provisioning, each with the value that won, plus an App mode line. If Path Logs says something other than what you expected, the override list directly above it names the layer responsible.
One quirk to expect on a package install: because the unit passes cfg:default.* arguments, you will see override lines on a completely stock system with an untouched grafana.ini. That is normal and not a sign anyone has been editing your config.
The default line format is logfmt, key and value separated by equals signs, which greps cleanly and parses into most log tools without a custom pattern. A startup line looks roughly like logger=sqlstore t=2026-08-26T09:14:22.118Z level=info msg="Connecting to DB" dbtype=sqlite3. The two fields you will use constantly are logger= to find the subsystem and level= to filter severity.
logfmt came out of Heroku and never got a formal specification, which is why parsers disagree about quoting and escaping at the edges. Grafana quotes consistently enough that it rarely bites. Anyone shipping several applications into one index will meet the problem eventually, at which point JSON is the answer.
If the logs are heading into Loki, Elasticsearch or anything else that would rather not reverse-engineer logfmt, switch the format:
[log.file]
format = jsonSet the same on [log.console] when you are shipping container stdout. Grafana emits proper JSON objects per line, so no multiline handling is needed and the logger field becomes a label you can filter on directly. This is the point at which reading Grafana's own logs turns into a monitoring exercise, and if you are going that far the same pipeline that watches a Node.js app with Prometheus and Grafana can watch Grafana itself.
Browser side logging with [log.frontend]
Grafana can also collect JavaScript errors from the browser and forward them, which is a different system from everything above. It is off by default:
[log.frontend]
enabled = false
custom_endpoint = /log-grafana-javascript-agent
log_endpoint_requests_per_second_limit = 3
log_endpoint_burst_limit = 15Turning it on sends frontend errors to a Grafana Faro collector at custom_endpoint, with rate limits so a broken dashboard on fifty screens cannot flood you. Unless you are developing plugins or chasing a rendering bug that only reproduces on one person's browser, leave it off.
Shipping Grafana's logs somewhere else
On a systemd host, point Promtail or Grafana Alloy at the journal unit and you are done, with no Grafana config change at all. In containers, keep console mode, set format = json and let the Docker logging driver forward stdout. Anything already running rsyslog gets the shortest path of the lot: add syslog to mode and set a facility.
The combination to avoid is a tail-based collector scraping the file while Grafana's own rotation is active. Rotation renames files under the collector, and you spend an afternoon working out why a chunk of lines went missing. Console mode into the journal, or JSON into a log driver, sidesteps it.
Once logs are readable, the errors in them become searchable, and most of what you will find is covered in the Grafana common errors and fixes reference. A permissions failure on /var/lib/grafana after a manual chown is the most common reason a package install writes one line to the log and then stops. Check ownership before anything else on a Grafana that will not start.
Two related settings live nearby in the same config file. router_logging = false in [server] turns on a line per HTTP request when you flip it, useful for exactly as long as it takes to prove a reverse proxy is forwarding what you think it is. And log_queries in [database] logs SQL calls with execution times, which overlaps with the sqlstore filter but includes timings. Both belong to the same family of switches you turn on for an afternoon and then turn back off, alongside the Grafana hardening settings you leave on permanently.
If you are standing up a fresh instance to test any of this against, installing Grafana on an Ubuntu VPS from the APT repo takes about four minutes, or the one-click Grafana VPS template skips the manual steps and hands you a running instance with the packaged layout described above. Both give you the deb paths.
Sources worth bookmarking: the Grafana configuration reference for every key in every section, the troubleshooting page for the documented log paths and the Docker installation docs for the image's baked-in environment. When the docs and a blog post disagree about a path, the docs are right and the blog post is running an old version.
