Setting memory limits

[1]:

import aopy

Available memory unused by anyone

[2]:

aopy.utils.get_memory_available_gb()

[2]:

Undoing the default memory limit

[3]:

aopy.utils.get_memory_limit_gb()

[3]:

[4]:

aopy.utils.release_memory_limit()

[5]:

aopy.utils.get_memory_limit_gb()

Using memory limits

[6]:

aopy.utils.set_memory_limit_gb(128)

[7]:

import numpy as np
a = np.ones((1000000,10000))
print(a.nbytes/1e9)

80.0

[8]:

b = np.ones((200000,10000))
print(b.nbytes/1e9)

16.0

[9]:

c = np.ones((200000,10000))
print((a.nbytes + b.nbytes + c.nbytes)/1e9)

112.0

[10]:

c -= b

[11]:

d = np.ones((200000,10000))

---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
Input In [15], in <cell line: 1>()
----> 1 d = np.ones((200000,10000))

File ~/miniconda3/envs/leo-analysis/lib/python3.9/site-packages/numpy/core/numeric.py:204, in ones(shape, dtype, order, like)
    201 if like is not None:
    202     return _ones_with_like(shape, dtype=dtype, order=order, like=like)
--> 204 a = empty(shape, dtype, order)
    205 multiarray.copyto(a, 1, casting='unsafe')
    206 return a

MemoryError: Unable to allocate 14.9 GiB for an array with shape (200000, 10000) and data type float64

[12]:

aopy.utils.release_memory_limit()

[13]:

d = np.ones((200000,10000))
print((a.nbytes + b.nbytes + c.nbytes + d.nbytes)/1e9)

128.0

[14]:

aopy.utils.get_memory_available_gb()

[14]:

[15]:

aopy.utils.release_memory_limit()

[16]:

aopy.utils.get_memory_available_gb()

[16]:

Seeing total memory

[17]:

!grep MemTotal /proc/meminfo

MemTotal:       1056466716 kB

[ ]:

[ ]: