Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from http://stackoverflow.com/questions/107705/python-output-buffering below:

python - Disable output buffering

Asked 16 years, 11 months ago

Viewed 430k times

Is output buffering enabled by default in Python's interpreter for sys.stdout?

If the answer is positive, what are all the ways to disable it?

Suggestions so far:

1. Use the -u command line switch
2. Wrap sys.stdout in an object that flushes after every write
3. Set PYTHONUNBUFFERED env var
4. sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

Is there any other way to set some global flag in sys/sys.stdout programmatically during execution?

_{If you just want to flush after a specific write using print, see How can I flush the output of the print function?.}

61.2k1414 gold badges131131 silver badges191191 bronze badges

asked Sep 20, 2008 at 9:17

275k9292 gold badges365365 silver badges426426 bronze badges

From Magnus Lycka answer on a mailing list:

You can skip buffering for a whole python process using python -u or by setting the environment variable PYTHONUNBUFFERED.

You could also replace sys.stdout with some other stream like wrapper which does a flush after every call.

class Unbuffered(object):
   def __init__(self, stream):
       self.stream = stream
   def write(self, data):
       self.stream.write(data)
       self.stream.flush()
   def writelines(self, datas):
       self.stream.writelines(datas)
       self.stream.flush()
   def __getattr__(self, attr):
       return getattr(self.stream, attr)

import sys
sys.stdout = Unbuffered(sys.stdout)
print 'Hello'

answered Sep 20, 2008 at 9:24

17.9k77 gold badges4040 silver badges2727 bronze badges

# reopen stdout file descriptor with write mode
# and 0 as the buffer size (unbuffered)
import io, os, sys
try:
    # Python 3, open as binary, then wrap in a TextIOWrapper with write-through.
    sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), 'wb', 0), write_through=True)
    # If flushing on newlines is sufficient, as of 3.7 you can instead just call:
    # sys.stdout.reconfigure(line_buffering=True)
except TypeError:
    # Python 2
    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

Credits: "Sebastian", somewhere on the Python mailing list.

8,59544 gold badges4242 silver badges4343 bronze badges

answered Oct 8, 2008 at 7:23

47.2k3131 gold badges112112 silver badges134134 bronze badges

Yes, it is.

You can disable it on the commandline with the "-u" switch.

Alternatively, you could call .flush() on sys.stdout on every write (or wrap it with an object that does this automatically)

answered Sep 20, 2008 at 9:25

120k2929 gold badges110110 silver badges114114 bronze badges

This relates to Cristóvão D. Sousa's answer, but I couldn't comment yet.

A straight-forward way of using the flush keyword argument of Python 3 in order to always have unbuffered output is:

import functools
print = functools.partial(print, flush=True)

afterwards, print will always flush the output directly (except flush=False is given).

Note, (a) that this answers the question only partially as it doesn't redirect all the output. But I guess print is the most common way for creating output to stdout/stderr in python, so these 2 lines cover probably most of the use cases.

Note (b) that it only works in the module/script where you defined it. This can be good when writing a module as it doesn't mess with the sys.stdout.

Python 2 doesn't provide the flush argument, but you could emulate a Python 3-type print function as described here https://stackoverflow.com/a/27991478/3734258 .

answered Oct 20, 2016 at 18:30

1,17222 gold badges99 silver badges1717 bronze badges

The following works in Python 2.6, 2.7, and 3.2:

import os
import sys
buf_arg = 0
if sys.version_info[0] == 3:
    os.environ['PYTHONUNBUFFERED'] = '1'
    buf_arg = 1
sys.stdout = os.fdopen(sys.stdout.fileno(), 'a+', buf_arg)
sys.stderr = os.fdopen(sys.stderr.fileno(), 'a+', buf_arg)

answered Apr 12, 2014 at 18:43

16111 silver badge22 bronze badges

def disable_stdout_buffering():
    # Appending to gc.garbage is a way to stop an object from being
    # destroyed.  If the old sys.stdout is ever collected, it will
    # close() stdout, which is not good.
    gc.garbage.append(sys.stdout)
    sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

# Then this will give output in the correct order:
disable_stdout_buffering()
print "hello"
subprocess.call(["echo", "bye"])

Without saving the old sys.stdout, disable_stdout_buffering() isn't idempotent, and multiple calls will result in an error like this:

Traceback (most recent call last):
  File "test/buffering.py", line 17, in <module>
    print "hello"
IOError: [Errno 9] Bad file descriptor
close failed: [Errno 9] Bad file descriptor

Another possibility is:

def disable_stdout_buffering():
    fileno = sys.stdout.fileno()
    temp_fd = os.dup(fileno)
    sys.stdout.close()
    os.dup2(temp_fd, fileno)
    os.close(temp_fd)
    sys.stdout = os.fdopen(fileno, "w", 0)

(Appending to gc.garbage is not such a good idea because it's where unfreeable cycles get put, and you might want to check for those.)

answered Sep 9, 2010 at 15:37

1,4821414 silver badges1111 bronze badges

In Python 3, you can monkey-patch the print function, to always send flush=True:

_orig_print = print

def print(*args, **kwargs):
    _orig_print(*args, flush=True, **kwargs)

As pointed out in a comment, you can simplify this by binding the flush parameter to a value, via functools.partial:

print = functools.partial(print, flush=True)

answered Nov 26, 2018 at 20:07

29.9k1111 gold badges8181 silver badges114114 bronze badges

Yes, it is enabled by default. You can disable it by using the -u option on the command line when calling python.

answered Sep 20, 2008 at 9:26

12.4k1212 gold badges5858 silver badges6363 bronze badges

You can also run Python with stdbuf utility:

stdbuf -oL python <script>

answered Jul 1, 2015 at 20:09

74066 silver badges1515 bronze badges

It is possible to override only write method of sys.stdout with one that calls flush. Suggested method implementation is below.

def write_flush(args, w=stdout.write):
    w(args)
    stdout.flush()

Default value of w argument will keep original write method reference. After write_flush is defined, the original write might be overridden.

stdout.write = write_flush

The code assumes that stdout is imported this way from sys import stdout.

answered Feb 25, 2017 at 21:00

6311 silver badge44 bronze badges

You can also use fcntl to change the file flags in-fly.

fl = fcntl.fcntl(fd.fileno(), fcntl.F_GETFL)
fl |= os.O_SYNC # or os.O_DSYNC (if you don't care the file timestamp updates)
fcntl.fcntl(fd.fileno(), fcntl.F_SETFL, fl)

answered Nov 15, 2009 at 0:01

1,02222 gold badges1212 silver badges1212 bronze badges

One way to get unbuffered output would be to use sys.stderr instead of sys.stdout or to simply call sys.stdout.flush() to explicitly force a write to occur.

You could easily redirect everything printed by doing:

import sys; sys.stdout = sys.stderr
print "Hello World!"

Or to redirect just for a particular print statement:

print >>sys.stderr, "Hello World!"

To reset stdout you can just do:

sys.stdout = sys.__stdout__

15k66 gold badges3434 silver badges3636 bronze badges

answered Sep 20, 2008 at 9:40

8,74211 gold badge3737 silver badges5151 bronze badges

You can create an unbuffered file and assign this file to sys.stdout.

import sys 
myFile= open( "a.log", "w", 0 ) 
sys.stdout= myFile

You can't magically change the system-supplied stdout; since it's supplied to your python program by the OS.

answered Sep 20, 2008 at 10:39

393k8383 gold badges518518 silver badges791791 bronze badges

Variant that works without crashing (at least on win32; python 2.7, ipython 0.12) then called subsequently (multiple times):

def DisOutBuffering():
    if sys.stdout.name == '<stdout>':
        sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

    if sys.stderr.name == '<stderr>':
        sys.stderr = os.fdopen(sys.stderr.fileno(), 'w', 0)

answered Jun 30, 2012 at 19:20

3911 bronze badge

(I've posted a comment, but it got lost somehow. So, again:)

1. As I noticed, CPython (at least on Linux) behaves differently depending on where the output goes. If it goes to a tty, then the output is flushed after each '\n'
   If it goes to a pipe/process, then it is buffered and you can use the flush() based solutions or the -u option recommended above.

2. Slightly related to output buffering:
   If you iterate over the lines in the input with

   for line in sys.stdin:
   ...

then the for implementation in CPython will collect the input for a while and then execute the loop body for a bunch of input lines. If your script is about to write output for each input line, this might look like output buffering but it's actually batching, and therefore, none of the flush(), etc. techniques will help that. Interestingly, you don't have this behaviour in pypy. To avoid this, you can use

while True: line=sys.stdin.readline()
...

answered Jun 11, 2013 at 14:47

55277 silver badges1111 bronze badges

Start asking to get answers

Find the answer to your question by asking.

Explore related questions

See similar questions with these tags.

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4