Permission denied to tor hidden service directory on CentOS after service restart

To Ka asked:

I have installed tor from EPEL on a fresh installed CentOS7 (minimal).

After configuring a hidden service I check and validate that the service is up and running by browsing the URL generated in hostname file from the tor browser.

At this point the service is configured to start automatically using systemctl enable tor

Everything works find until I restart the service sudo systemctl restart tor. After that tor service does not star anymore logging the following error:
Directory /var/lib/tor/hidden_service_01/ cannot be read: Permission denied

hidden_service_01 folder is automatically created by the tor service when it runs the first time.

if I delete hidden_service_01 folder and start the service again. It starts up (generating a new .onion url). But once it’s stopped and started again the permission error occurs again.

Why am I getting the permission error and how to I make it work?

P.S. I created a guide snippet the I used for my configuration:
https://gist.github.com/Dzoge/f059d30da77a21df1a0f29a0b5c528a2

UPDATE 1:
I have checked the folder permissions and they were:

  • folder permissions are rwx------. 2 toranon toranon
  • two files inside the folder are rw-------. 1 toranon toranon

I set chmod to 770 and now folder and files all have rwxrwx---. 1 toranon toranon

I try to start tor service systemctl start tor and still get the same permission warning.

UPDATE 2:
I tried doing the same on Ubuntu server and it worked pretty well. I updated the gist as well and added a guide for ubuntu.

The problem still remains with CentOS 7.

UPDATE 3 (workaround):
Looks like a temporary workaround is to set SELinux to permissive mode. Then it works as expected.

My answer:


If you take a look at the tor.service unit, you will see that it has a command to attempt to verify the Tor configuration before starting the service.

ExecStartPre=/usr/bin/tor --runasdaemon 0 --defaults-torrc /usr/share/tor/defaults-torrc -f /etc/tor/torrc --verify-config

This is where things are going wrong. When this runs, the configuration fails to validate due to the permission problem noted earlier.

Jan 06 16:18:42 dalaran systemd[1]: Starting Anonymizing overlay network for TCP...
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.650 [notice] Tor 0.2.9.14 (git-3f9bd01bf5736ff6) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2k-fips and Zlib 1.2.7.
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.650 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.650 [notice] Read configuration file "/usr/share/tor/defaults-torrc".
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.650 [notice] Read configuration file "/etc/tor/torrc".
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.653 [warn] Directory /var/lib/tor/hidden_service_01/ cannot be read: Permission denied
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.653 [warn] Checking service directory /var/lib/tor/hidden_service_01/ failed.
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.653 [warn] Failed to parse/validate config: Failed to configure rendezvous options. See logs for details.
Jan 06 16:18:42 dalaran tor[28731]: Jan 06 16:18:42.653 [err] Reading config failed--see warnings above.
Jan 06 16:18:42 dalaran systemd[1]: tor.service: control process exited, code=exited status=1
Jan 06 16:18:42 dalaran systemd[1]: Failed to start Anonymizing overlay network for TCP.
Jan 06 16:18:42 dalaran systemd[1]: Unit tor.service entered failed state.
Jan 06 16:18:42 dalaran systemd[1]: tor.service failed.

Eventually I traced this down to some of the hardening that systemd does here. If you’ll read further in the unit file, you’ll see that systemd actually runs Tor in a container, and locks down the permissions very tightly. It also removes some capabilities, so that even root cannot do some actions the root user would normally be able to do, such as read other users’ files (this is known as CAP_DAC_OVERRIDE).

When we attempt to audit the failed start, we find:

type=PATH msg=audit(1515277122.651:3600): item=0 name="/var/lib/tor/hidden_service_01/" inode=1054341 dev=fd:01 mode=040700 ouid=988 ogid=983 rdev=00:00 obj=system_u:object_r:tor_var_lib_t:s0 objtype=NORMAL
type=CWD msg=audit(1515277122.651:3600):  cwd="/"
type=SYSCALL msg=audit(1515277122.651:3600): arch=c000003e syscall=2 success=no exit=-13 a0=561b6881af10 a1=20000 a2=0 a3=1 items=1 ppid=1 pid=28731 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="tor" exe="/usr/bin/tor" subj=system_u:system_r:tor_t:s0 key=(null)
type=AVC msg=audit(1515277122.651:3600): avc:  denied  { dac_read_search } for  pid=28731 comm="tor" capability=2  scontext=system_u:system_r:tor_t:s0 tcontext=system_u:system_r:tor_t:s0 tclass=capability
type=AVC msg=audit(1515277122.651:3600): avc:  denied  { dac_override } for  pid=28731 comm="tor" capability=1  scontext=system_u:system_r:tor_t:s0 tcontext=system_u:system_r:tor_t:s0 tclass=capability

What I found here is that the command to verify config was not actually changing from the root user to the toranon user before reading the configuration, so access to the directory was being denied, as systemd did not give the process CAP_DAC_OVERRIDE. You can see the user IDs were logged as uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0.

So to resolve this on my system, I decided to have systemd start Tor as toranon rather than starting as root and Tor changing its own uid/gid.

  1. I removed the User toranon setting from /usr/share/tor/torrc-defaults.
  2. I created an override file /etc/systemd/system/tor.service.d/override.conf containing:

    [Service]
    User=toranon
    Group=toranon
    PermissionsStartOnly=no
    

It’s necessary to use PermissionsStartOnly=no to ensure that the ExecStartPre= command is run under the toranon user. From the documentation:

PermissionsStartOnly=
Takes a boolean argument. If true, the permission-related execution options, as configured with User= and similar options (see systemd.exec(5) for more information), are only applied to the process started with ExecStart=, and not to the various other ExecStartPre=, ExecStartPost=, ExecReload=, ExecStop=, and ExecStopPost= commands. If false, the setting is applied to all configured commands the same way. Defaults to false.

After a systemctl daemon-reload I was then able to systemctl start tor successfully.

Jan 06 16:22:02 dalaran systemd[1]: Starting Anonymizing overlay network for TCP...
Jan 06 16:22:02 dalaran tor[32030]: Jan 06 16:22:02.541 [notice] Tor 0.2.9.14 (git-3f9bd01bf5736ff6) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2k-fips and Zlib 1.2.7.
Jan 06 16:22:02 dalaran tor[32030]: Jan 06 16:22:02.541 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jan 06 16:22:02 dalaran tor[32030]: Jan 06 16:22:02.541 [notice] Read configuration file "/usr/share/tor/defaults-torrc".
Jan 06 16:22:02 dalaran tor[32030]: Jan 06 16:22:02.541 [notice] Read configuration file "/etc/tor/torrc".
Jan 06 16:22:02 dalaran tor[32030]: Configuration was valid
Jan 06 16:22:02 dalaran systemd[1]: Started Anonymizing overlay network for TCP.
Jan 06 16:22:02 dalaran tor[32035]: Jan 06 16:22:02.665 [notice] Tor 0.2.9.14 (git-3f9bd01bf5736ff6) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2k-fips and Zlib 1.2.7.
Jan 06 16:22:02 dalaran tor[32035]: Jan 06 16:22:02.665 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jan 06 16:22:02 dalaran tor[32035]: Jan 06 16:22:02.665 [notice] Read configuration file "/usr/share/tor/defaults-torrc".
Jan 06 16:22:02 dalaran tor[32035]: Jan 06 16:22:02.665 [notice] Read configuration file "/etc/tor/torrc".
Jan 06 16:22:02 dalaran tor[32035]: Jan 06 16:22:02.668 [notice] Opening Socks listener on 127.0.0.1:9050
Jan 06 16:22:02 dalaran tor[32035]: Jan 06 16:22:02.668 [notice] Opening Control listener on /run/tor/control
Jan 06 16:22:02 dalaran Tor[32035]: OpenSSL version from headers does not match the version we're running with. If you get weird crashes, that might be why. (Compiled with 100020bf: OpenSSL 1.0.2k  26 Jan 2017; running wit
Jan 06 16:22:02 dalaran Tor[32035]: Tor 0.2.9.14 (git-3f9bd01bf5736ff6) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2k-fips and Zlib 1.2.7.
Jan 06 16:22:02 dalaran Tor[32035]: Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jan 06 16:22:02 dalaran Tor[32035]: Read configuration file "/usr/share/tor/defaults-torrc".
Jan 06 16:22:02 dalaran Tor[32035]: Read configuration file "/etc/tor/torrc".
Jan 06 16:22:02 dalaran Tor[32035]: Opening Socks listener on 127.0.0.1:9050
Jan 06 16:22:02 dalaran Tor[32035]: Opening Control listener on /run/tor/control
Jan 06 16:22:02 dalaran Tor[32035]: Parsing GEOIP IPv4 file /usr/share/tor/geoip.
Jan 06 16:22:02 dalaran Tor[32035]: Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
Jan 06 16:22:02 dalaran Tor[32035]: Bootstrapped 0%: Starting
Jan 06 16:22:02 dalaran Tor[32035]: Bootstrapped 80%: Connecting to the Tor network
Jan 06 16:22:03 dalaran Tor[32035]: Bootstrapped 85%: Finishing handshake with first hop
Jan 06 16:22:04 dalaran Tor[32035]: Bootstrapped 90%: Establishing a Tor circuit
Jan 06 16:22:05 dalaran Tor[32035]: Tor has successfully opened a circuit. Looks like client functionality is working.
Jan 06 16:22:05 dalaran Tor[32035]: Bootstrapped 100%: Done

Ultimately this (or something like it) needs to be incorporated into the Tor default config and systemd unit shipped by Fedora/EPEL, which will resolve the problem for everyone.


View the full question and answer on Server Fault.

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.