Search code examples
pythonlistpython-3.xpipgenerator

No module named mem_profile


I'm using this program to measure the time and memory used by two functions and compare which is better for processing a large amount of data. My understanding is that to measure the memory usage we need the mem_profile module, but during the pip install mem_profile it gave me the error No module named mem_profile.

import mem_profile
import random
import time

names = ['Kiran','King','John','Corey']
majors = ['Math','Comps','Science']

print 'Memory (Before): {}Mb'.format(mem_profile.memory_usage_resource())

def people_list(num_people):
    results = []
    for i in num_people:
        person = {
                    'id':i,
                    'name': random.choice(names),
                    'major':random.choice(majors)
                  }
        results.append(person)
    return results

def people_generator(num_people):
    for i in xrange(num_people):
        person = {
                    'id':i,
                    'name': random.choice(names),
                    'major':random.choice(majors)
                  }
        yield person

t1 = time.clock()
people = people_list(10000000)
t2 = time.clock()


# t1 = time.clock()
# people = people_generator(10000000)
# t2 = time.clock()

print 'Memory (After): {}Mb'.format(mem_profile.memory_usage_resource())
print 'Took {} Seconds'.format(t2-t1)

What has caused this error? And are there any alternative packages I could use instead?


Solution

  • Use this for calculating time:

    import time
    
    time_start = time.time()
    #run your code
    time_elapsed = (time.time() - time_start)
    

    As referenced by the Python documentation:

    time.time() → float Return the time in seconds since the epoch as a floating point number. The specific date of the epoch and the handling of leap seconds is platform dependent. On Windows and most Unix systems, the epoch is January 1, 1970, 00:00:00 (UTC) and leap seconds are not counted towards the time in seconds since the epoch. This is commonly referred to as Unix time. To find out what the epoch is on a given platform, look at gmtime(0).

    Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second. While this function normally returns non-decreasing values, it can return a lower value than a previous call if the system clock has been set back between the two calls.

    The number returned by time() may be converted into a more common time format (i.e. year, month, day, hour, etc…) in UTC by passing it to gmtime() function or in local time by passing it to the localtime() function. In both cases a struct_time object is returned, from which the components of the calendar date may be accessed as attributes.

    Reference: https://docs.python.org/3/library/time.html#time.time


    Use this for calculating memory:

    import resource
    
    resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    

    Reference: http://docs.python.org/library/resource.html

    Use this if you using python 3.x:

    Reference: https://docs.python.org/3/library/timeit.html