[ Python 文章收集 ] 5 must-know Python concepts for experts

Source From Here
Preface
Python is a really powerful programming language of the modern era. To code in Python is as easy as implementing pseudo code and as verbose as the English language. On top of it, python boasts of a large repository of modules for just about any problem that the programmer wants to tinker with and a large community base supporting issue-resolution, discussions, best-practices and compatibility. No wonder, it is one of the top programming languages in the world today and a language of choice for working in certain domains such as Web development, Data Science & Machine Learning due to some powerful frameworks like Django, Flask, Tornado, Dash etc.

Yet, after everything said, once you actually start programming in Python, you discover that there are much more powerful features, hidden inside this deceptively simple language to handle tricky situations and common pitfalls with ease. For e.g. Python allows the programmer to use the ‘else’ keyword with loops; just as it is conventionally used with the ‘if’ keyword. Imagine a problem where you evaluate each element of a list to check if the list contains an even number or not. Using a for-else combination, you can write an implementation as simple as one given below; no need to use a Boolean variable to mark whether the even number was found or not in the list:

view plaincopy to clipboardprint?

1. #!/usr/bin/env python3  
2. # Sample function to process the even number  
3. def processEven(number):  
4.     print("Even number found: {}".format(number))  
5.   
6. # The for-else loop  
7. for listOfNumbers in [[1,2,3,4,5], [1,3,5]]:  
8.     for number in listOfNumbers:  
9.         if number % 2 == 0:  
10.             processEven(number)  
11.             break  
12.     else:  
13.         print("An even number was not found in the list={} !".format(listOfNumbers))  

Output:

Even number found: 2
An even number was not found in the list=[1, 3, 5] !

This kind of implementation is extremely useful in initializing default values to return from a loop if a break statement is not encountered during the iterations. But this is only the tip of the iceberg. Here are the 5 advanced concepts to know in Python.

CONTEXT MANAGERS
Ever opened a file to perform a read/write operation? Of course you have. And did you remember to close the file and release the file-handle, after the operation has been done? Mostly! Enters Context Managers. Context Managers allow the programmer to open and close the file or connection object using a ‘[b]with’ and an ‘as’ keyword. It automatically takes cares of the object after execution has finished[/b]; making sure that the connection or file object is safely released or closed afterwards. The object is safely closed even when an error occurs while executing the processing logic within the block. So, mostly it replaces this code:
someFile = open(‘some_file’, ‘w’)

view plaincopy to clipboardprint?

1. try:  
2.     someFile.write(‘Hello World !’)  
3. finally:  
4.     someFile.close()  

with this code:

view plaincopy to clipboardprint?

1. with open(‘some_file’, ‘w’) as someFile:  
2.     someFile.write(‘Hello World !’)  

Simple, easy and secure!

IMPLICIT TUPLE UNPACKING
Python supports this feature to allow functions like multiple assignments and multiple returns in a single statement that makes the life of the programmer easier. Multiple assignments means that this is possible:

view plaincopy to clipboardprint?

1. x, y = 10, 20  

Herein, Python is implicitly creating a tuple (10, 20) out of the supplied values and iterating over the variables supplied for individual assignment. The creation of a temporary tuple also means that a copy of the value supplied is used, and if the r_values are variables (e.g. x, y = amount, name), it behaves like ‘pass-by-value’ technique. This means you can do this without creating a race-condition:

view plaincopy to clipboardprint?

1. x, y = y, x  

There goes the smug interview question about swapping two variables without using a third one. But, that’s not all. This functionality can be extended over different data types. Doing this:

view plaincopy to clipboardprint?

1. x, y = ‘OK’  

results in x=’O’ and y=’K’. Python also allows you to return multiple values from a function without the need to define a ‘structure’, populating all values in its object and returning the object. You can simple do this:

view plaincopy to clipboardprint?

1. def function():  
2.     # Some processing  
3.     return name, salary, employeeID  
4. x, y, z = function()  

This is the feature of the language which allows you to automatically unpack a elements in a list over a loop:

view plaincopy to clipboardprint?

1. list = [(‘Alice’,25), (‘Bob’,30), (‘Jake’,27), (‘Barbara’,40)]  
2. for name, age in list:  
3.     print(name,’is aged’,str(age))  

MAGIC METHODS
Magic methods are implicitly invoked functions when certain operations occur on the object of a particular class. They are surrounded by double-underscores and each can be defined while creating your own class to easily impart certain properties to it. Consider the following code sample:

view plaincopy to clipboardprint?

1. #!/usr/bin/env python3  
2. class Employee:  
3.     def __init__(self, name, ID, salary):  
4.         self.empName = name  
5.         self.empID = ID  
6.         self.empSalary = salary  
7.     def __str__(self):  
8.         return f"Employee({self.empName}, {self.empID}, {self.empSalary})"  
9.     def __repr__(self):  
10.         return f"[ {self.empID} ] - {self.empName}"  
11.     def __add__(self, secondObject):  
12.         return (self.empSalary + secondObject.empSalary)  
13.   
14.   
15. if __name__ == "__main__":  
16.     objAlice = Employee("Alice", "EMP001", 10000)  
17.     print(objAlice)  
18.     print(repr(objAlice))  
19.     print("")  
20.     objBob = Employee("Bob", "EMP002", 5000)  
21.     print(objBob)  
22.     print(repr(objBob))  
23.     print("\nSum: {}".format(objAlice+objBob))  

Output:

Employee(Alice, EMP001, 10000)
[ EMP001 ] - Alice

Employee(Bob, EMP002, 5000)
[ EMP002 ] - Bob

Sum: 15000

As is visible, the __str__() method is implicitly invoked when print() is called on the object. The __repr__() method defines a representation of the class object however it makes sense. The method __add__() allows you to define what happens when the ‘+’ operator is used with the objects of the class. The __init__() method is like a constructor of the class.

In short, magic methods lets the programmer define what happens when some of the common operators and functions are used on the object. Without the __add__() method defined in the above example, the interpreter won’t know what to do when two objects of the class are added together. Using the magic methods within the class definition, the programmer can control its behavior when used with common operators.

GENERATORS
Generators are lazy iterators that process an element in a list only when it is used. Consider a function that processes a very large list. Normally, the large list needs to be loaded in memory (in a container, a variable) before another function can process it. This means very large data corpus pose a space-complexity problem for the program. Imagine, instead a technique that loads the data only when it is their turn to be processed and not before that. That technique is called Generators in python. Consider the following code:

view plaincopy to clipboardprint?

1. #!/usr/bin/env python3  
2. import sys  
3. import time  
4. # Normal function  
5. def getList(limit):  
6.     listVal = list()  
7.     for i in range(limit):  
8.         listVal.append(i)  
9.     return listVal  
10.   
11. # Generator function  
12. def genList(limit):  
13.     for i in range(limit):  
14.         yield i  
15.   
16. if __name__ == "__main__":  
17.     numLimit = 10000000  
18.     print("\nWithout Generators:")  
19.     startTime = time.time()  
20.     numList = getList(numLimit)  
21.     usedTime = time.time() - startTime  
22.     usedMem = sys.getsizeof(numList)  
23.     print(f" — Time: {usedTime} seconds")  
24.     print(f" — Size: {usedMem}")  
25.     print("\nWith Generators:")  
26.     startTime = time.time()  
27.     numGenerator = genList(numLimit)  
28.     usedTime = time.time() - startTime  
29.     usedMem = sys.getsizeof(numGenerator )  
30.     print(f" — Time: {usedTime} seconds")  
31.     print(f" — Size: {usedMem}")  

Output:

Without Generators:
— Time: 0.7022416591644287 seconds
— Size: 81528056

With Generators:
— Time: 2.6226043701171875e-06 seconds
— Size: 88

Notice that in the normal function generates the list and returns the value to ‘numList’ variable, which would take up a lot of memory till the processing is complete. This becomes a serious problem while processing say a very large corpus of files. The second function though, doesn’t generates the list right away. The list elements would be ‘generated’ one-by-one as they need to be processed and hence, the time& space-complexity of the program remains low; even when processing Big Data.

Another big advantage of the generators is that at the ‘yield’ statement, control is passed back from the function to the calling program and the state of the local variables are remembered for the next iteration. This means that if you need to conditionally look for, say, prime numbers in the generated stream and stop processing when consecutive 3 numbers are detected which are not prime, you don’t need to have loaded a very large list from a file of 1000 numbers. Using Generators, you can load the elements one-by-one, process them till 3 consecutive non-primes appear, and then terminate the program.

DECORATORS
In Python, functions are objects; meaning that they can be passed as argument and returned from another functions. Decorators take advantage of this feature and provide a method to wrap functions inside another to impart additional functionalities without changing the behavior of the original function. Let me explain this with a use-case. Imagine you have written a program that does a lot of time taking operations like loading a large file, making an API call, generating a summary report etc. After you have written everything, you wish to calculate the time it takes for each of these operations. The most common method to do that is to use the time module as shown below:

view plaincopy to clipboardprint?

1. import time  
2.   
3. if __name__ == ‘__main__’:  
4.     startTime = time.time()  
5.     loadLargeFile("abc.txt")  
6.     usedTime = time.time() - startTime  
7.     print(f"Time: {usedTime} seconds’)  

All well and good. But what if this was to be done for many methods? You would need to add 3 additional lines of code around each function call across your program, which would be a lot of effort to truly cover all function calls. And what if after the analysis, you no longer want to do the same? Go through the code base and remove these additional lines all over again to undo what you did. There has to be a better method. There is — Decorators. Consider the code below:

view plaincopy to clipboardprint?

1. #!/usr/bin/env python3  
2. import time  
3.   
4. def timeIt(func):  
5.     def wrapper(*args, **kwargs):  
6.         startTime = time.time()  
7.         func(*args, **kwargs)  
8.         usedTime = time.time() - startTime  
9.         print(f"Time: {usedTime} seconds")  
10.     return wrapper  
11.   
12. @timeIt  
13. def loadLargeFile(filename):  
14.     print(f"Loading file: {filename}")  
15.     time.sleep(2)  
16.   
17. @timeIt  
18. def makeAPICall():  
19.     print("Making an API call and waiting for the response...")  
20.     time.sleep(1.5)  
21.   
22. @timeIt  
23. def generateSummaryReport():  
24.     print("Generating summary report...")  
25.     time.sleep(5)  
26.   
27. if __name__ == "__main__":  
28.     loadLargeFile("abc.txt")  
29.     makeAPICall()  
30.     generateSummaryReport()  

Output:

Loading file: abc.txt
Time: 2.0022945404052734 seconds
Making an API call and waiting for the response...
Time: 1.5018024444580078 seconds
Generating summary report...
Time: 5.0017194747924805 seconds

As you can notice, this defines some functions (I am mocking long operations with a ‘sleep’ call) and a wrapper method — timeIt(), that has the code to calculate the time of the passed function object. Just by adding ‘@timeIt’ before the defined functions, one can wrap the function call inside the timeIt() wrapper. This is equivalent to doing:

view plaincopy to clipboardprint?

1. timeIt(loadLargeFile(‘abc.txt’))  
2. timeIt(makeAPICall())  
3. timeIt(generateSummaryReport())  

And when I no longer need to wrap the functions, I can just go to their definitions and remove the preceding ‘@timeIt’ to reverse the ‘decoration’. Not just this; python supports multiple decorators to be applied to the same function at the same time. In that case, wrapping would occur in the order in which decorators are applied.

I have mentioned some of the hidden features of the Python language in this post and tried to explain them with the help of some sample codes. But Python boasts of a lot of other strengths apart from the ones mentioned above. I strongly suggest reading up a little on the official documentation to uncover these. Using the advanced concepts like Generators and Decorators can actually mean shipping cleaner and highly-maintainable code which is also free from errors.

Supplement
* 如何使用 Python 進行字串格式化