2018年12月31日 星期一

[ Python 常見問題 ] Text Progress Bar in the Console

Source From Here 
Question 
As title. How to draw a progress bar for long time execution task in command line mode? 

How-To 
For a hand-made version: 
  1. # Print iterations progress  
  2. def printProgressBar (iteration, total, prefix = '', suffix = '', decimals = 1, length = 100, fill = '█'):  
  3.     """  
  4.     Call in a loop to create terminal progress bar  
  5.     @params:  
  6.         iteration   - Required  : current iteration (Int)  
  7.         total       - Required  : total iterations (Int)  
  8.         prefix      - Optional  : prefix string (Str)  
  9.         suffix      - Optional  : suffix string (Str)  
  10.         decimals    - Optional  : positive number of decimals in percent complete (Int)  
  11.         length      - Optional  : character length of bar (Int)  
  12.         fill        - Optional  : bar fill character (Str)  
  13.     """  
  14.     percent = ("{0:." + str(decimals) + "f}").format(100 * (iteration / float(total)))  
  15.     filledLength = int(length * iteration // total)  
  16.     bar = fill * filledLength + '-' * (length - filledLength)  
  17.     print('\r%s |%s| %s%% %s' % (prefix, bar, percent, suffix), end = '\r')  
  18.     # Print New Line on Complete  
  19.     if iteration == total:   
  20.         print()  
  21.   
  22. #   
  23. # Sample Usage  
  24. #   
  25.   
  26. from time import sleep  
  27.   
  28. # A List of Items  
  29. items = list(range(0, 57))  
  30. l = len(items)  
  31.   
  32. # Initial call to print 0% progress  
  33. printProgressBar(0, l, prefix = 'Progress:', suffix = 'Complete', length = 50)  
  34. for i, item in enumerate(items):  
  35.     # Do stuff...  
  36.     sleep(0.1)  
  37.     # Update Progress Bar  
  38.     printProgressBar(i + 1, l, prefix = 'Progress:', suffix = 'Complete', length = 50)  
For a exist python package tqdm: add a progress meter to your loops in a second: 
  1. >>> import time  
  2. >>> from tqdm import tqdm  
  3. >>> for i in tqdm(range(100)):  
  4. ...     time.sleep(1)  
  5. ...  
  6. 27%|██████████████████████████████████                                                                                            | 27/100 [00:36<01:18,  1.08s/it]  


2018年12月25日 星期二

[ Python 常見問題 ] Is there a simple process-based parallel map for python?

Source From Here 
Question 
I'm looking for a simple process-based parallel map for python. With native support map function, the performance: 
  1. In [1]: data = range(10000000)  
  2.   
  3. In [2]: time alist = list(map(lambda e:(e*5+1)/2, data))  
  4. CPU times: user 1.48 s, sys: 47.6 ms, total: 1.53 s  
  5. Wall time: 1.53 s  
  6.   
  7. In [3]: time olist = [(e*5+1)/2 for e in data]  
  8. CPU times: user 862 ms, sys: 54 ms, total: 916 ms  
  9. Wall time: 917 ms  
How-To 
I seems like what you need is the map method in multiprocessing.Pool()
map(func, iterable[, chunksize])

A parallel equivalent of the map() built-in function (it supports only one iterable argument though). It blocks till the result is ready. This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer

Below is the sample code to show the usage: 
- test.py 
  1. #!/usr/bin/env python3  
  2. import multiprocessing  
  3. from datetime import datetime  
  4.   
  5. def f(e):  
  6.     return (e*5+1)/2  
  7.   
  8. data = range(10000000)  
  9. pool = multiprocessing.Pool()  
  10. st = datetime.now()  
  11. print("Start at {}".format(st))  
  12. mlist = pool.map(f, data)  
  13. diff = datetime.now() - st  
  14. print("Done with {} ms".format(diff.microseconds/1000))  
Execution result: 
$ ./test.py
Start at 2018-12-25 22:11:46.570080
Done with 245.617 ms

Supplement 
Python 文章收集 - multiprocessing 模塊介紹

2018年12月24日 星期一

[ Python 常見問題 ] Currying decorator in python

Source From Here 
Question 
I am trying to write a currying decorator as @curry in python so the function can be: 
  1. @curry  
  2. def myfun(a,b,c):  
  3.     print("{}-{}-{}".format(a,b,c))  
  4.       
  5. myfun(123)  
  6. myfun(12)(3)  
  7. myfun(1)(2)(3)   
How-To 
If you are using Python2: 
  1. def curry(x, argc=None):  
  2.     if argc is None:  
  3.         argc = x.func_code.co_argcount  
  4.     def p(*a):  
  5.         if len(a) == argc:  
  6.             return x(*a)  
  7.         def q(*b):  
  8.             return x(*(a + b))  
  9.         return curry(q, argc - len(a))  
  10.     return p  
  11.   
  12. @curry  
  13. def myfun(a,b,c):  
  14.     print '%d-%d-%d' % (a,b,c)  
In Python3: 
  1. from inspect import signature  
  2.   
  3. def curry(x, argc=None):  
  4.     if argc is None:  
  5.         argc = len(signature(x).parameters)  
  6.           
  7.     def p(*a):  
  8.         if len(a) == argc:  
  9.             return x(*a)  
  10.         def q(*b):  
  11.             return x(*(a + b))  
  12.         return curry(q, argc - len(a))  
  13.     return p  
  14.   
  15. @curry  
  16. def myfun(a,b,c):  
  17.     print("{}-{}-{}".format(a,b,c))  
  18.       
  19. myfun(123)  
  20. myfun(12)(3)  
  21. myfun(1)(2)(3)  
Output: 
1-2-3
1-2-3
1-2-3

Supplement 
How can I find the number of arguments of a Python function?

[ Py DS ] Ch5 - Machine Learning (Part1)

Source From  Here   What Is Machine Learning?   Before we take a look at the details of various machine learning methods, let’s start by ...