I'm using this program to measure the time and memory used by two functions and compare which is better for processing a large amount of data. My understanding is that to measure the memory usage we need the mem_profile
module, but during the pip install mem_profile
it gave me the error No module named mem_profile
.
import mem_profile
import random
import time
names = ['Kiran','King','John','Corey']
majors = ['Math','Comps','Science']
print 'Memory (Before): {}Mb'.format(mem_profile.memory_usage_resource())
def people_list(num_people):
results = []
for i in num_people:
person = {
'id':i,
'name': random.choice(names),
'major':random.choice(majors)
}
results.append(person)
return results
def people_generator(num_people):
for i in xrange(num_people):
person = {
'id':i,
'name': random.choice(names),
'major':random.choice(majors)
}
yield person
t1 = time.clock()
people = people_list(10000000)
t2 = time.clock()
# t1 = time.clock()
# people = people_generator(10000000)
# t2 = time.clock()
print 'Memory (After): {}Mb'.format(mem_profile.memory_usage_resource())
print 'Took {} Seconds'.format(t2-t1)
What has caused this error? And are there any alternative packages I could use instead?
Use this for calculating time:
import time
time_start = time.time()
#run your code
time_elapsed = (time.time() - time_start)
As referenced by the Python documentation:
time.time() → float Return the time in seconds since the epoch as a floating point number. The specific date of the epoch and the handling of leap seconds is platform dependent. On Windows and most Unix systems, the epoch is January 1, 1970, 00:00:00 (UTC) and leap seconds are not counted towards the time in seconds since the epoch. This is commonly referred to as Unix time. To find out what the epoch is on a given platform, look at gmtime(0).
Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls.
The number returned by time() may be converted into a more common time format (i.e. year, month, day, hour, etc…) in UTC by passing it to gmtime() function or in local time by passing it to the localtime() function. In both cases a struct_time object is returned, from which the components of the calendar date may be accessed as attributes.
Reference: https://docs.python.org/3/library/time.html#time.time
Use this for calculating memory:
import resource
resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
Reference: http://docs.python.org/library/resource.html
Use this if you using python 3.x: