Python – Use Twisted to run commands on remote systems

I’m trying to write a client/server using Twisted that allows the client to issue remote commands on the server and receive response data in real time. That is, if I run $> ssh server someProg.sh, I will see the results “live” instead of seeing all the results immediately when the process completes. Is this kind of thing possible in Twisted? Thank you.

Of course. As already pointed out in the comments, you can > by to do it. This is more scalable (you can open a lot of connections without any additional processes) and more portable (it works on Windows), but it doesn’t take your OpenSSH configuration into account, and you have to write a bunch of extra code to handle things like host key validation. Another issue that doesn’t directly address your main issue either, it’s about what was being handled when the output arrived.

The simple answer is “is”, but here’s a demo program that spawns multiple child processes and displays their output. You can replace sys.executable with another program you want to build, i.e. ssh, which works exactly the same.

import os, sys

from twisted.internet.protocol import ProcessProtocol
from twisted.internet import reactor
from twisted.internet.defer import Deferred, gatherResults

script = """
import time
for x in range(3):
    time.sleep(1)
    print(x)
"""

class SimpleProcess(ProcessProtocol):
    def __init__(self, id, d):
        self.id = id
        self.d = d
    def outReceived(self, out):
        print('Received output: {out} from: {proc}'
              .format(out=repr(out), proc=self.id))
    def processEnded(self, reason):
        self.d.callback(None)

ds = []
for x in range(3):
    d = Deferred()
    reactor.callLater(
        x * 0.5, reactor.spawnProcess, SimpleProcess(x, d),
        sys.executable, [sys.executable, '-u', '-c', script],
        os.environ)
    ds.append(d)

gatherResults(ds).addBoth(lambda ignored: reactor.stop())

reactor.run()