How To Disable Stdout Buffer When Running Shell

I am using Python to call a Shell script with def run_command(cmd): print 'Start to run: ' + cmd run = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=sub

Solution 1:

The buffering in question would largely be a problem on the script's side, not the Python side; while Python would buffer the reads, it wouldn't block unless the buffer was emptied and there was nothing available to read.

So really, you need to disable buffering in the script itself. Adding stdbuf -oL (or -o0 for completely unbuffered, but line buffering should cover you since you read by line as well) to your commands should help in some cases (where the programs don't adjust their own buffering internally).

If you're seeing this behavior only by looking at Python's output, be aware that Python itself can buffer output as well. You can disable this by passing -u when running Python, or setting the environment variable PYTHONUNBUFFERED=1 before running it, or from within a script, you can manually call sys.stdout.flush() after any writes (direct, or implicit via print) to stdout. On modern Python, print takes an argument to force a flush after printing, but since you're on Python 2.x, that's not an option.