pythonparallel-processingjoblib# A Python function did not work correctly when run on multiple cores, but runs normally on a single core

This function did not work correctly when `n_jobs!=1`

```
from joblib import Parallel, delayed
import numpy as np
import pandas as pd
def foo(n_jobs):
result = {}
x = np.sin(4 * np.pi * np.arange(0, 1, 0.001))
y = np.sin(8 * np.pi * np.arange(0, 1, 0.001) + np.pi/2)
x2yT = np.zeros(x.shape[0])
y2xT = np.zeros(x.shape[0])
def parallelize(ite):
xi = x[ite] * 2
yi = y[ite] + 1
y2xT[ite] = xi
x2yT[ite] = yi
r = Parallel(n_jobs=n_jobs)(delayed(parallelize)(i) for i in np.arange(x.shape[0]))
result[f'y2xT'] = y2xT
result[f'x2yT'] = x2yT
return pd.DataFrame(result)
```

The following code snippet produces a plot using a function `foo`

with a single core:

```
r0 = foo( n_jobs = 1)
r0.plot()
```

The resulting plot is shown below:

The following code snippet produces a plot using a function `foo`

with multiple cores:

```
r0 = foo( n_jobs = -1)
r0.plot()
```

The resulting plot is shown below:

How can I ensure that the function utilizes multiple cores correctly

Solution

According to the comment from Frank, I tried to convert `y2xT`

and `x2yT`

from `numpy.ndarray`

to `numpy.memmap`

, and it works!

```
def foo(n_jobs):
result = {}
x = np.sin(4 * np.pi * np.arange(0, 1, 0.001))
y = np.sin(8 * np.pi * np.arange(0, 1, 0.001) + np.pi/2)
# x2yT = np.zeros(x.shape[0])
# y2xT = np.zeros(x.shape[0])
x2yT = np.memmap('x2yT.pkl', dtype=np.float32, shape=x.shape[0], mode='w+')
y2xT = np.memmap('y2xT.pkl', dtype=np.float32, shape=x.shape[0], mode='w+')
def parallelize(ite):
xi = x[ite] * 2
yi = y[ite] + 1
y2xT[ite] = xi
x2yT[ite] = yi
r = Parallel(n_jobs=n_jobs)(delayed(parallelize)(i) for i in np.arange(x.shape[0]))
result[f'y2xT'] = y2xT
result[f'x2yT'] = x2yT
return pd.DataFrame(result)
r0 = foo( n_jobs = -1)
r0.plot()
```

- Python Jinja2 LaTeX Table
- Getting attributes of a class
- How can I print many significant figures in Python?
- How to allow list append() method to return the new list
- Calculate Last Friday of Month in Pandas
- Python type hint for Iterable[str] that isn't str
- How to iterate over a list in chunks
- How to exit the entire application from a Python thread?
- Running shell command and capturing the output
- How do I pass a variable by reference?
- Convert range(r) to list of strings of length 2 in python
- How can I get the start and end dates for each week?
- how to use send_message() in python-telegram-bot
- Python conditional replacement based on element type
- How can I count the number of items in an arbitrary iterable (such as a generator)?
- Find longest consecutive range of numbers in list
- Insert text in braces with asyncpg
- How does one put a link / url to the web-site's home page in Django?
- How to determine if a path is a subdirectory of another?
- Custom Keybindings for Ipython terminal
- FastAPI asynchronous background tasks blocks other requests?
- How to make sure that information from one file is duplicated into several text documents, without specific lines
- Installing a Python environment with Anaconda
- sklearn pipeline model predicting same results for all input
- Brew command not found after installing Anaconda Python
- How to get an XPath from selenium webelement or from lxml?
- Pipe PuTTY console to Python script
- How to align the axes of a figure in matplotlib?
- Persist ParentDocumentRetriever of langchain
- How to reset index in a pandas dataframe?