Home - News ( United States | United Kingdom | Italy | Germany ) - Football scores

Showing content from https://github.com/pandas-dev/pandas/issues/54654 below:

ensure_string_array may change the array inplace · Issue #54654 · pandas-dev/pandas · GitHub

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

import numpy as np
import pickle
from pandas._libs.lib import ensure_string_array
t = np.array([None, None, {'a': 1.235}])
tp = pickle.loads(pickle.dumps(t))
g = np.asarray(tp, dtype='object')

assert g is not tp
assert np.shares_memory(g, tp)

print(tp) # Expected [None, None, {'a': 1.235}]
print(ensure_string_array(tp)) # got [nan, nan, "{'a': 1.235}"]
print(tp) # Expected [None, None, {'a': 1.235}], got [nan, nan, "{'a': 1.235}"]

I've had an issue where df.astype(str) actually changed df in-place instead of only making a copy.
I've looked into where the values mutate and I found out that it originates in the cython implementation of ensure_string_array.

• the validation on result is only using arr is not result to determine whether to copy the data
• while this condition may be false for result = np.asarray(arr, dtype='object') - it may still share memory
  verified with np.shares_memory(arr, result)
• I suggest to add/alter the condition with np.shares_memory.

I tried to make a code that replicates the issue with the astype() example, but I didn't manage to do it easily.
I can also make a PR if the solution sounds good to you.

BTW, I'm not sure why the pickle loads/dumps change the behaviour here, but the example doesn't work without it

astype() should return a copy, if not specified otherwise
ensure_string_array() should return a copy, if not specified otherwise

commit : 37ea63d
python : 3.10.11.final.0
python-bits : 64
OS : Linux
OS-release : 6.2.0-26-generic
Version : #26~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Jul 13 16:27:29 UTC 2
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 2.0.1
numpy : 1.24.3
pytz : 2023.3
dateutil : 2.8.2
setuptools : 57.5.0
pip : 22.3.1
Cython : 0.29.36
pytest : 7.3.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : 3.1.2
lxml.etree : 4.9.3
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.13.2
pandas_datareader: None
bs4 : 4.12.2
bottleneck : 1.3.7
brotli :
fastparquet : None
fsspec : 2023.6.0
gcsfs : 2023.6.0
matplotlib : 3.7.2
numba : 0.57.1
numexpr : 2.8.4
odfpy : None
openpyxl : 3.1.2
pandas_gbq : 0.17.9
pyarrow : 9.0.0
pyreadstat : None
pyxlsb : 1.0.10
s3fs : None
scipy : 1.10.1
snappy :
sqlalchemy : None
tables : 3.8.0
tabulate : None
xarray : 2023.6.0
xlrd : 2.0.1
zstandard : 0.21.0
tzdata : 2023.3
qtpy : None
pyqt5 : None

RetroSearch is an open source project built by @garambo | Open a GitHub Issue

Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo

HTML: 3.2 | Encoding: UTF-8 | Version: 0.7.4