memoryview in Python

Python is a language that is well known for its speed and performance. That is, while working with large iterations or files, we need to optimize the code to boost the speed and performance. There memoryview objects comes into picture.

The memoryview uses the buffer protocol to access the binary object's memory without actually copying. A memoryview object comes into the category of binary sequence type like bytes and bytearray.

What is memoryview in Python ?

A memoryview is a generalized NumPy array structure. It provides to share the memory between data structures without first copying. It plays an important role to optimize the efficiency and performance of the program when working with large data sets.

Why memoryview in Python ?

The main objective of memoryview in Python is, it allows to access the internal data of an object. Here the object must support the buffer protocol. Objects that supports buffer protocol in Python are bytes and bytearray.

Note - A memoryview object is useful when slicing the data without copying the underlying data.

How do we Create a memoryview Object in Python ?

To create a memoryview object in Python, use the memoryview() constructor as shown in the syntax given below:

memoryview(obj)

Here obj is an object. This object must support the buffer protocol. bytes and bytearray are the two inbuilt objects, that support the buffer protocol.

Python memoryview Example

Here is a very simple program, that shows the benefits of using the memoryview object in Python.

import time

n = 500000
data = b'x' * n
b = data
start = time.time()
while b:
    b = b[1:]
tm = time.time() - start
print("Time taken for", n, "iterations (without memoryview):", tm, "sec")

n = 500000
data = b'x' * n
b = memoryview(data)
start = time.time()
while b:
    b = b[1:]
tm = time.time() - start
print("Time taken for", n, "iterations (with memoryview):", tm, "sec")

The output :

That is, it takes 4.9095458984375 seconds when iterated for 500000 times without using the memoryview object. And takes only 0.07810783386230469 seconds when iterated for again 500000 times, but with memoryview object.

Let's create another program in Python that uses the similar approach as of above program, to check the number of times taken to iterate a loop (while loop) for some large number of times, without using memoryview object. After this program, I've created another program, with memoryview object.

import time

for n in (10000, 20000, 30000, 40000, 90000):
    bData = b'codescracker'*n
    start = time.time()
    while bData:
        bData = bData[1:]
    print("Time taken when n is", n, ":", time.time() - start)

The snapshot given below shows the sample output produced by this Python program:

In the above program, first the value 10000 gets initialized to n. And inside the block of for loop, the binary object b'codescracker'*10000 or 10000 times codescracker gets initialized to bData variable as binary object. Now the current time is initialized to start variable.

And the execution of while loop initiated. This loop continues its execution until the last character of bData variable. The bData[1:] refers to all the characters except the very first character. Meaning that, the while loop will get iterated for n or 10000 number of times for first iteration of for loop. Similarly, 20000 number of times when the value of n will be 20000, or for second iteration of for loop.

After executing all the iterations of while loop, the time taken to execute all the iteration will get calculated using:

time.time() - start

and printed using the print() statement. See the last time, that was 50 seconds. For that, I've to wait for 50 seconds to complete the last iteration. Multiplying "codescracker" with 90000 gives total of 12*90000 characters. And to iterate each and every character one by one of total 1080000 characters will takes pretty much time when we go with normal approach. But let's do the same thing with memoryview object.

Here is the same program to iterate the while loop with memoryview object to calculate and print the time taken for the iterations:

import time

for n in (10000, 20000, 30000, 40000, 90000):
    bData = b'codescracker'*n
    bData = memoryview(bData)
    start = time.time()
    while bData:
        bData = bData[1:]
    print("Time taken when n is", n, ":", time.time() - start)

See how the iteration becomes fast, when the bData (that has to be iterated) gets converted into a memoryview object. Here is the combined version of both the above programs with some little modifications:

import time

print("Type\t\t\tIterations\t\tTime Taken")
print("-------------------------------------------")

for n in (100000, 200000, 300000, 400000, 500000):
    data = b'x'*n
    start = time.time()
    b = data
    while b:
        b = b[1:]
    end = time.time()
    tm = end - start
    print(f'bytes \t\t\t{n}\t\t\t {tm:0.2f}')

print()

for n in (100000, 200000, 300000, 400000, 500000):
    data = b'x'*n
    start = time.time()
    b = memoryview(data)
    while b:
        b = b[1:]
    end = time.time()
    tm = end - start
    print(f'memoryview  \t{n}\t\t\t {tm:0.2f}')

Here is its sample output:

I don't know how you utilize the memoryview object in your application. But the thing is, this object uses the buffer protocol to access the memory of binary object without actually copying.

And I think the concept of memoryview object is clearly described using the notes and example given above. Now let's create an example that prints the value and the type of a memoryview type variable.

x = memoryview(b'codescracker')
print(x)
print(type(x))

The output produces by above Python program, demonstrating the memoryview object, looks similar to:

<memory at 0x000001BD90E81700>
<class 'memoryview'>

Here is another example on memoryview, allows user to input the data:

print("Enter the Size: ", end="")
n = int(input())
print("Enter", n, "Strings: ", end="")
mystr = []
for i in range(n):
    val = input()
    mystr.append(val)

print("\nValue      MemoryView")
print("----------------------")
for i in range(n):
    x = bytes(mystr[i], "utf-16")
    print(mystr[i], "    ", memoryview(x))

The snapshot given below shows the sample run of above program, with user input 4 as size, codes, cracker, dot, com as four string:

Here is another sample run with some other user inputs: