Search code examples
pythonpandasdataframenumpygetlocation

Python: How to get position of pandas.series element where conditions exist?


I have a numpy.ndarrays: x,y:

>>> x = np.ndarray(shape=(10,), buffer=np.array([0.9902, 0.9394, 0.839,  0.8574, 0.9174, 0.8742, 0.8955, 0.9196, 0.9388, 0.9602]), dtype=float)

[0.9902 0.9394 0.839  0.8574 0.9174 0.8742 0.8955 0.9196 0.9388 0.9602]

>>> y = np.ndarray(shape=(10,), buffer=np.array([0.956, 0.884, 0.875, 0.880, 0.865, 0.870, 0.861, 0.817, 0.771, 0.727]), dtype=float)

[0.956, 0.884, 0.875, 0.880, 0.865, 0.870, 0.861, 0.817, 0.771, 0.727]

and series edge_or_not:

>>> d = {'2020-03-17 04:39:00+03:00': 0,
          '2020-03-17 04:40:00+03:00': 1,
          '2020-03-17 04:41:00+03:00': 0,
          '2020-03-17 04:42:00+03:00': -1,
          '2020-03-17 04:43:00+03:00': 0,
          '2020-03-17 04:44:00+03:00': 0,
          '2020-03-17 04:45:00+03:00': 1,
          '2020-03-17 04:46:00+03:00': -1,
          '2020-03-17 04:47:00+03:00': -1,
          '2020-03-17 04:48:00+03:00': -1}

>>> edge_or_not = pd.Series(data=d)

2020-03-17 04:39:00+03:00    0
2020-03-17 04:40:00+03:00    1
2020-03-17 04:41:00+03:00    0
2020-03-17 04:42:00+03:00   -1
2020-03-17 04:43:00+03:00    0
2020-03-17 04:44:00+03:00    0
2020-03-17 04:45:00+03:00    1
2020-03-17 04:46:00+03:00   -1
2020-03-17 04:47:00+03:00   -1
2020-03-17 04:48:00+03:00   -1
dtype: int64

And I'm getting up_edge_x, up_edge_y, down_edge_x, down_edge_y like this:

>>> up_edge_x = x[edge_or_not > 0]

array([0.9394, 0.8955])

>>> up_edge_y = y[edge_or_not > 0]

array([0.884, 0.861])

>>> down_edge_x = x[edge_or_not < 0]

array([0.8574, 0.9196, 0.9388, 0.9602])

>>> down_edge_y = y[edge_or_not < 0]

array([0.88 , 0.817, 0.771, 0.727])

And all_edges_x, all_edges_y:

>>> all_edges_x = x[edge_or_not != 0]

array([0.9394, 0.8574, 0.8955, 0.9196, 0.9388, 0.9602])

>>> all_edges_y = y[edge_or_not != 0]

array([0.884, 0.88 , 0.861, 0.817, 0.771, 0.727])

And then creating DataFrames:

>>> up_edge = pd.DataFrame({'y':up_edge_y}, index=up_edge_x)

            y   (pos)
0.9394  0.884       0
0.8955  0.861       1

>>> down_edge = pd.DataFrame({'y':down_edge_y}, index=down_edge_x)

            y   (pos)
0.8574  0.880       0
0.9196  0.817       1
0.9388  0.771       2
0.9602  0.727       3

All I need is creating all_edges DataFrame where will be 3 columns: 'y', 'edge', 'pos'

>>> all_edges = pd.DataFrame({'y':all_edges_y, 'edge':edge_or_not[edge_or_not != 0].to_numpy(), 
                         'pos':???},
                          index=all_edges_x)

So that after all all_edges DataFrame must look like this:

            y  edge  pos
0.9394  0.884     1    0
0.8574  0.880    -1    0
0.8955  0.861     1    1
0.9196  0.817    -1    1
0.9388  0.771    -1    2
0.9602  0.727    -1    3

How to calculate 3rd column pos, that I can links to all_edges from up_edge and down_edge DataFrames like in below stupid example:

>>> down_x1 = 0.9602
>>> loc = down_edge.index.get_loc(down_x1)
>>> edges = all_edges.loc[all_edges['pos']==loc]['edge']
>>> print(edges)

0.9602   -1
Name: edge, dtype: int64

And I've got a second question: How to get array of locations another DataFrame? Like this:

>>> locations = down_edge.index.get_loc(#mb all indexes)

[0, 1, 2, 3]

Solution

  • Use:

    up_edge_x = x[edge_or_not > 0]
    up_edge_y = y[edge_or_not > 0]
    
    down_edge_x = x[edge_or_not < 0]
    down_edge_y = y[edge_or_not < 0]
    
    all_edges_x = x[edge_or_not != 0]
    all_edges_y = y[edge_or_not != 0]
    

    Create Series by ranges with index by up_edge_x, down_edge_x first:

    up_edge = pd.Series(range(len(up_edge_x)), index=up_edge_x, name='pos')
    down_edge = pd.Series(range(len(down_edge_x)), index=down_edge_x, name='pos')
    print (up_edge)
    0.9394    0
    0.8955    1
    Name: pos, dtype: int64
    
    print (down_edge)
    0.8574    0
    0.9196    1
    0.9388    2
    0.9602    3
    Name: pos, dtype: int64
    

    Then join together:

    pos = pd.concat([up_edge, down_edge])
    print (pos)
    0.9394    0
    0.8955    1
    0.8574    0
    0.9196    1
    0.9388    2
    0.9602    3
    Name: pos, dtype: int64
    

    And last map new column:

    all_edges = pd.DataFrame({'y':all_edges_y,
                              'edge':edge_or_not[edge_or_not != 0].to_numpy(), 
                              'pos': pd.Index(all_edges_x).map(pos)},
                              index=all_edges_x)
    
    
    print (all_edges)
                y  edge  pos
    0.9394  0.884     1    0
    0.8574  0.880    -1    0
    0.8955  0.861     1    1
    0.9196  0.817    -1    1
    0.9388  0.771    -1    2
    0.9602  0.727    -1    3