2018年12月31日 星期一

[ Python 常見問題 ] Text Progress Bar in the Console

Source From Here 
Question 
As title. How to draw a progress bar for long time execution task in command line mode? 

How-To 
For a hand-made version: 
  1. # Print iterations progress  
  2. def printProgressBar (iteration, total, prefix = '', suffix = '', decimals = 1, length = 100, fill = '█'):  
  3.     """  
  4.     Call in a loop to create terminal progress bar  
  5.     @params:  
  6.         iteration   - Required  : current iteration (Int)  
  7.         total       - Required  : total iterations (Int)  
  8.         prefix      - Optional  : prefix string (Str)  
  9.         suffix      - Optional  : suffix string (Str)  
  10.         decimals    - Optional  : positive number of decimals in percent complete (Int)  
  11.         length      - Optional  : character length of bar (Int)  
  12.         fill        - Optional  : bar fill character (Str)  
  13.     """  
  14.     percent = ("{0:." + str(decimals) + "f}").format(100 * (iteration / float(total)))  
  15.     filledLength = int(length * iteration // total)  
  16.     bar = fill * filledLength + '-' * (length - filledLength)  
  17.     print('\r%s |%s| %s%% %s' % (prefix, bar, percent, suffix), end = '\r')  
  18.     # Print New Line on Complete  
  19.     if iteration == total:   
  20.         print()  
  21.   
  22. #   
  23. # Sample Usage  
  24. #   
  25.   
  26. from time import sleep  
  27.   
  28. # A List of Items  
  29. items = list(range(0, 57))  
  30. l = len(items)  
  31.   
  32. # Initial call to print 0% progress  
  33. printProgressBar(0, l, prefix = 'Progress:', suffix = 'Complete', length = 50)  
  34. for i, item in enumerate(items):  
  35.     # Do stuff...  
  36.     sleep(0.1)  
  37.     # Update Progress Bar  
  38.     printProgressBar(i + 1, l, prefix = 'Progress:', suffix = 'Complete', length = 50)  
For a exist python package tqdm: add a progress meter to your loops in a second: 
  1. >>> import time  
  2. >>> from tqdm import tqdm  
  3. >>> for i in tqdm(range(100)):  
  4. ...     time.sleep(1)  
  5. ...  
  6. 27%|██████████████████████████████████                                                                                            | 27/100 [00:36<01:18,  1.08s/it]  


2018年12月25日 星期二

[ Python 常見問題 ] Is there a simple process-based parallel map for python?

Source From Here 
Question 
I'm looking for a simple process-based parallel map for python. With native support map function, the performance: 
  1. In [1]: data = range(10000000)  
  2.   
  3. In [2]: time alist = list(map(lambda e:(e*5+1)/2, data))  
  4. CPU times: user 1.48 s, sys: 47.6 ms, total: 1.53 s  
  5. Wall time: 1.53 s  
  6.   
  7. In [3]: time olist = [(e*5+1)/2 for e in data]  
  8. CPU times: user 862 ms, sys: 54 ms, total: 916 ms  
  9. Wall time: 917 ms  
How-To 
I seems like what you need is the map method in multiprocessing.Pool()
map(func, iterable[, chunksize])

A parallel equivalent of the map() built-in function (it supports only one iterable argument though). It blocks till the result is ready. This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer

Below is the sample code to show the usage: 
- test.py 
  1. #!/usr/bin/env python3  
  2. import multiprocessing  
  3. from datetime import datetime  
  4.   
  5. def f(e):  
  6.     return (e*5+1)/2  
  7.   
  8. data = range(10000000)  
  9. pool = multiprocessing.Pool()  
  10. st = datetime.now()  
  11. print("Start at {}".format(st))  
  12. mlist = pool.map(f, data)  
  13. diff = datetime.now() - st  
  14. print("Done with {} ms".format(diff.microseconds/1000))  
Execution result: 
$ ./test.py
Start at 2018-12-25 22:11:46.570080
Done with 245.617 ms

Supplement 
Python 文章收集 - multiprocessing 模塊介紹

2018年12月24日 星期一

[ Python 常見問題 ] Currying decorator in python

Source From Here 
Question 
I am trying to write a currying decorator as @curry in python so the function can be: 
  1. @curry  
  2. def myfun(a,b,c):  
  3.     print("{}-{}-{}".format(a,b,c))  
  4.       
  5. myfun(123)  
  6. myfun(12)(3)  
  7. myfun(1)(2)(3)   
How-To 
If you are using Python2: 
  1. def curry(x, argc=None):  
  2.     if argc is None:  
  3.         argc = x.func_code.co_argcount  
  4.     def p(*a):  
  5.         if len(a) == argc:  
  6.             return x(*a)  
  7.         def q(*b):  
  8.             return x(*(a + b))  
  9.         return curry(q, argc - len(a))  
  10.     return p  
  11.   
  12. @curry  
  13. def myfun(a,b,c):  
  14.     print '%d-%d-%d' % (a,b,c)  
In Python3: 
  1. from inspect import signature  
  2.   
  3. def curry(x, argc=None):  
  4.     if argc is None:  
  5.         argc = len(signature(x).parameters)  
  6.           
  7.     def p(*a):  
  8.         if len(a) == argc:  
  9.             return x(*a)  
  10.         def q(*b):  
  11.             return x(*(a + b))  
  12.         return curry(q, argc - len(a))  
  13.     return p  
  14.   
  15. @curry  
  16. def myfun(a,b,c):  
  17.     print("{}-{}-{}".format(a,b,c))  
  18.       
  19. myfun(123)  
  20. myfun(12)(3)  
  21. myfun(1)(2)(3)  
Output: 
1-2-3
1-2-3
1-2-3

Supplement 
How can I find the number of arguments of a Python function?

[Git 文章收集] Differences between git merge and git rebase

Source From  Here Preface Merging and rebasing are the two most popular way to applying changes from one branch into another one. They bot...