Python Tutorial: Generators - How to use them and the benefits you receive
Why Stack Overflow recommends it
Stack Overflow users link it when answering questions about memory use or slow loops, as a clear introduction to generators for handling large data lazily.
Recommended by 3 people across 2 different months, 2018 to 2020.
What it is
A Corey Schafer tutorial on Python generators: what they are, how the yield keyword works, and why they are more readable and memory-efficient than building full lists. It walks through code examples and compares performance, with the code available on GitHub.
What people said
Recommends Tekay37 · 2018-01 · Stack Overflow · on “Insert output back into Nested For Loop Python 3”
I did learn python generators via Corey Schafers youtube videos: [this video] And there is a PyCon talk about generators on youtube.
Recommends Franco D'Angelo · 2018-01 · Stack Overflow · on “Project Euler Q3 Python Very Slow”
You can also use generators [this video] , that worked for me. Here is an example: # Set variables number = 600851475143 primeFactorList = [] def primeList(number): # Make list of prime numbers < 'number' for x in range(2, number+1): isPrime = True # Don't calculate for more than the sqrt of number for efficiency for y in range(2, int(x**0.5)+1): if x % y == 0: isPrime = False break if isPrime: yield x # Calculate primes using primeList and check for prime factors of 'number' for i in primeList(number): if i > number**0.5: break if number % i == 0: primeFactorList.append(i) # Print largest prime factor of 'number' print(max(primeFactorList))
Recommends Shaunak Sen · 2020-05 · Stack Overflow · on “Handling large list of lists in python”
Use generators for large data ranges, time and space complexity of the code will not increase exponentially with large data size, refer to the link for more details: [this video]
Cites Kishore Mohanavelu · 2020-09 · Stack Overflow · on “Not enough space Error appears when running for loop for 13K pdf documents”
There is a generator function in python, which can hold a large number of documents without having any impact on memory. You can try using the same. I am not sure whether your code is in python. Even if it is not in python, you can incorporate python library and execute only this piece of code in python. There is also a difflib library in python, which can compare documents with a single line of code. Please refer the below video for the same. [this video]