Python – In python, redis-py is used with multiprocessing modules, why is each process a different fd?

In python, redis-py is used with multiprocessing modules, why is each process a different fd?… here is a solution to the problem.

In python, redis-py is used with multiprocessing modules, why is each process a different fd?

In python, using redis-py with multiprocessing modules, why is each process a different fd?

Test code:

    # xiaorui.cc

import time
import multiprocessing

import redis

r = redis. Redis(host='127.0.0.1', port=6379, db=0)

def func(msg):
    for i in xrange(30):
        time.sleep(1)
        print r.keys()
    return "done " + msg

if __name__ == "__main__":
    pool = multiprocessing. Pool(processes=4)
    result = []
    for i in xrange(4):
        msg = "hello %d" %(i)
        result.append(pool.apply_async(func, (msg, )))
    pool.close()
    pool.join()
    for res in result:
        print res.get()
    print "Sub-process(es) done."

Test results:

[[email protected] ~]$ ps aux f|grep a.py
508      11704  5.0  0.0 421096 11664 pts/11   Sl+  17:50   0:00  |           \_ python a.py
508      11709  0.0  0.0 193760  7464 pts/11   S+   17:50   0:00  |               \_ python a.py
508      11710  0.0  0.0 193760  7468 pts/11   S+   17:50   0:00  |               \_ python a.py
508      11711  0.0  0.0 193760  7468 pts/11   S+   17:50   0:00  |               \_ python a.py
508      11712  0.0  0.0 193760  7476 pts/11   S+   17:50   0:00  |               \_ python a.py
508      11720  0.0  0.0 103248   832 pts/12   S+   17:50   0:00              \_ grep a.py
[[email protected] ~]$ sudo lsof -p 11709|grep 6379
python  11709 ruifengyun    4u  IPv4 4173927407      0t0        TCP localhost:51433->localhost:6379 (ESTABLISHED)
[[email protected] ~]$ sudo lsof -p 11710|grep 6379
python  11710 ruifengyun    4u  IPv4 4173927417      0t0        TCP localhost:51435->localhost:6379 (ESTABLISHED)
[[email protected] ~]$ sudo lsof -p 11711|grep 6379
python  11711 ruifengyun    4u  IPv4 4173927411      0t0        TCP localhost:51434->localhost:6379 (ESTABLISHED)
[[email protected] ~]$ sudo lsof -p 11712|grep 6379
python  11712 ruifengyun    4u  IPv4 4173927416      0t0        TCP localhost:51436->localhost:6379 (ESTABLISHED)

Why is the Redis Conn FD different for each process? In my opinion, different Redis FDs will only occur if Redis Connect is created in lazy mode.

Also, all child processes are shared R objects (Redis Conect FD).

Solution

Each process needs to connect to Redis separately. And a single outbound port will only be used by a single process. So each process has its own port.

Related Problems and Solutions