forked from Mirror/pmbootstrap
Properly escape commands in pmb.chroot.user() (#1316)
## Introduction In #1302 we noticed that `pmb.chroot.user()` does not escape commands properly: When passing one string with spaces, it would pass them as two strings to the chroot. The use case is passing a description with a space inside to `newapkbuild` with `pmboostrap newapkbuild`. This is not a security issue, as we don't pass strings from untrusted input to this function. ## Functions for running commands in pmbootstrap To put the rest of the description in context: We have four high level functions that run commands: * `pmb.helpers.run.user()` * `pmb.helpers.run.root()` * `pmb.chroot.root()` * `pmb.chroot.user()` In addition, one low level function that the others invoke: * `pmb.helpers.run.core()` ## Flawed test case The issue described above did not get detected for so long, because we have a test case in place since day one, which verifies that all of the functions above escape everything properly: * `test/test_shell_escape.py` So the test case ran a given command through all these functions, and compared the result each time. However, `pmb.chroot.root()` modified the command variable (passed by reference) and did the escaping already, which means `pmb.chroot.user()` running directly afterwards only returns the right output when *not* doing any escaping. Without questioning the accuracy of the test case, I've escaped commands and environment variables with `shlex.quote()` *before* passing them to `pmb.chroot.user()`. In retrospective this does not make sense at all and is reverted with this commit. ## Environment variables By coincidence, we have only passed custom environment variables to `pmb.chroot.user()`, never to the other high level functions. This only worked, because we did not do any escaping and the passed line gets executed as shell command: ``` $ MYENV=test echo test2 test 2 ``` If it was properly escaped as one shell command: ``` $ 'MYENV=test echo test2' sh: MYENV=test echo test2: not found ``` So doing that clearly doesn't work anymore. I have added a new `env` parameter to `pmb.chroot.user()` (and to all other high level functions for consistency), where environment variables can be passed as a dictionary. Then the function knows what to do and we end up with properly escaped commands and environment variables. ## Details * Add new `env` parameter to all high level command execution functions * New `pmb.helpers.run.flat_cmd()` function, that takes a command as list and environment variables as dict, and creates a properly escaped flat string from the input. * Use that function for proper escaping in all high level exec funcs * Don't escape commands *before* passing them to `pmb.chroot.user()` * Describe parameters of the command execution functions * `pmbootstrap -v` writes the exact command to the log that was executed (in addition to the simplified form we always write down for readability) * `test_shell_escape.py`: verify that the command passed by reference has not been modified, add a new test for strings with spaces, add tests for new function `pmb.helpers.run.flat_cmd()` * Remove obsolete commend in `pmb.chroot.distccd` about environment variables, because we don't use any there anymore * Add `TERM=xterm` to default environment variables in the chroot, so running ncurses applications like `menuconfig` and `nano` works out of the box
This commit is contained in:
Oliver Smith
GitHub
parent
571ddf741a
commit
3666388619
10 changed files with 236 additions and 87 deletions
|
|
@ -16,6 +16,7 @@ GNU General Public License for more details.
|
|||
You should have received a copy of the GNU General Public License
|
||||
along with pmbootstrap. If not, see <http://www.gnu.org/licenses/>.
|
||||
"""
|
||||
import shlex
|
||||
import subprocess
|
||||
import logging
|
||||
import os
|
||||
|
|
@ -23,19 +24,34 @@ import os
|
|||
|
||||
def core(args, cmd, log_message, log, return_stdout, check=True,
|
||||
working_dir=None, background=False):
|
||||
logging.debug(log_message)
|
||||
"""
|
||||
Run the command and write the output to the log.
|
||||
|
||||
:param cmd: command as list, e.g. ["echo", "string with spaces"]
|
||||
:param log_message: simplified and more readable form of the command, e.g.
|
||||
"(native) % echo test" instead of the full command with
|
||||
entering the chroot and more escaping
|
||||
:param log: * True: write stdout and stderr of the running process into
|
||||
the log file (read with "pmbootstrap log").
|
||||
* False: redirect stdout and stderr to pmbootstrap stdout
|
||||
:param return_stdout: write stdout to a buffer and return it as string when
|
||||
the command is through
|
||||
:param check: raise an exception, when the command fails
|
||||
:param working_dir: path in host system where the command should run
|
||||
:param background: run the process in the background and return the process
|
||||
handler
|
||||
:returns: * stdout when return_stdout is True
|
||||
* process handler when background is True
|
||||
* None otherwise
|
||||
"""
|
||||
logging.debug(log_message)
|
||||
logging.verbose("run: " + str(cmd))
|
||||
|
||||
if working_dir:
|
||||
working_dir_old = os.getcwd()
|
||||
os.chdir(working_dir)
|
||||
|
||||
ret = None
|
||||
|
||||
if background:
|
||||
if log:
|
||||
ret = subprocess.Popen(cmd, stdout=args.logfd, stderr=args.logfd)
|
||||
|
|
@ -72,23 +88,77 @@ def core(args, cmd, log_message, log, return_stdout, check=True,
|
|||
return ret
|
||||
|
||||
|
||||
def user(args, cmd, log=True, working_dir=None, return_stdout=False,
|
||||
check=True, background=False):
|
||||
def flat_cmd(cmd, working_dir=None, env={}):
|
||||
"""
|
||||
Convert a shell command passed as list into a flat shell string with
|
||||
proper escaping.
|
||||
|
||||
:param cmd: command as list, e.g. ["echo", "string with spaces"]
|
||||
:param working_dir: when set, prepend "cd ...;" to execute the command
|
||||
in the given working directory
|
||||
:param env: dict of environment variables to be passed to the command, e.g.
|
||||
{"JOBS": "5"}
|
||||
:returns: the flat string, e.g.
|
||||
echo 'string with spaces'
|
||||
cd /home/pmos;echo 'string with spaces'
|
||||
"""
|
||||
# Merge env and cmd into escaped list
|
||||
escaped = []
|
||||
for key, value in env.items():
|
||||
escaped.append(key + "=" + shlex.quote(value))
|
||||
for i in range(len(cmd)):
|
||||
escaped.append(shlex.quote(cmd[i]))
|
||||
|
||||
# Prepend working dir
|
||||
ret = " ".join(escaped)
|
||||
if working_dir:
|
||||
msg = "% cd " + working_dir + " && " + " ".join(cmd)
|
||||
else:
|
||||
msg = "% " + " ".join(cmd)
|
||||
ret = "cd " + shlex.quote(working_dir) + ";" + ret
|
||||
|
||||
# TODO: maintain and check against a whitelist
|
||||
return ret
|
||||
|
||||
|
||||
def user(args, cmd, log=True, working_dir=None, return_stdout=False,
|
||||
check=True, background=False, env={}):
|
||||
"""
|
||||
Run a command on the host system as user.
|
||||
|
||||
:param cmd: command as list, e.g. ["echo", "string with spaces"]
|
||||
:param log: when set to true, redirect all output to the logfile
|
||||
:param working_dir: path in host system where the command should run
|
||||
:param return_stdout: write stdout to a buffer and return it as string when
|
||||
the command is through
|
||||
:param check: raise an exception, when the command fails
|
||||
:param background: run the process in the background and return the process
|
||||
handler
|
||||
:param env: dict of environment variables to be passed to the command, e.g.
|
||||
{"JOBS": "5"}
|
||||
:returns: * stdout when return_stdout is True
|
||||
* process handler when background is True
|
||||
* None otherwise
|
||||
"""
|
||||
# Readable log message (without all the escaping)
|
||||
msg = "% "
|
||||
for key, value in env.items():
|
||||
msg += key + "=" + value + " "
|
||||
if working_dir:
|
||||
msg += "cd " + working_dir + "; "
|
||||
msg += " ".join(cmd)
|
||||
|
||||
# Add environment variables and run
|
||||
if env:
|
||||
cmd = ["sh", "-c", flat_cmd(cmd, env=env)]
|
||||
return core(args, cmd, msg, log, return_stdout, check, working_dir,
|
||||
background)
|
||||
|
||||
|
||||
def root(args, cmd, log=True, working_dir=None, return_stdout=False,
|
||||
check=True, background=False):
|
||||
check=True, background=False, env={}):
|
||||
"""
|
||||
:param working_dir: defaults to args.work
|
||||
Run a command on the host system as root, with sudo.
|
||||
|
||||
NOTE: See user() above for parameter descriptions.
|
||||
"""
|
||||
if env:
|
||||
cmd = ["sh", "-c", flat_cmd(cmd, env=env)]
|
||||
cmd = ["sudo"] + cmd
|
||||
return user(args, cmd, log, working_dir, return_stdout, check, background)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue