I'm new to the Python requests module and I'm trying to export game data from a user from the Lichess.org API (a chess website).
Here is my code:
import requests, json
url = "https://www.lichess.org/api/games/user/mbellm"
r = requests.get(url, params={"max":2, "analysed":"true", "clocks":"true", "evals":"true", "opening":"true"})
r_text = r.content.decode("utf-8"))
print(r_text)
#data = json.loads(r_text)
The output received in r_text
however seems to not be in JSON/NDJSON format, but rather in PGN format (a type of chess notation). In the section I linked to the API documentation above, it states that you can specify whether to receive data in JSON or PGN format but I don't see anywhere where it says how to choose the format which you receive it in.
How can I set my code to return the fetched data in JSON/NDJSON format?
Current output from print(r_text)
:
[Event "Rated Bullet game"]
[Site "https://lichess.org/IqlbjkHX"]
[Date "2019.01.13"]
[Round "-"]
[White "mbellm"]
[Black "Ruediruempel"]
[Result "0-1"]
[UTCDate "2019.01.13"]
[UTCTime "22:45:48"]
[WhiteElo "1097"]
[BlackElo "1202"]
[WhiteRatingDiff "-8"]
[BlackRatingDiff "+7"]
[Variant "Standard"]
[TimeControl "60+0"]
[ECO "C50"]
[Termination "Normal"]
1. e4 e5 2. Nf3 Nf6 3. Bc4 Nc6 4. d3 Bc5 5. Nc3 Nd4 6. O-O O-O 7. Nxd4 Bxd4 8. Nd5 Nxd5 9. Bxd5 c6 10. c3 cxd5 11. cxd4 dxe4 12. Qg4 exd3 13. dxe5 d6 14. Bd2 Bxg4 15. exd6 Qxd6 16. f3 Bf5 17. Rad1 h6 18. a3 Bh7 19. Bb4 Qd4+ 20. Kh1 Qxb2 21. Rxd3 Bxd3 22. Rc1 Qxc1+ 23. Be1 Qxe1# 0-1
[Event "Rated Classical game"]
[Site "https://lichess.org/sgNWdvkn"]
[Date "2019.01.13"]
[Round "-"]
[White "Stalingrad_1"]
[Black "mbellm"]
[Result "1-0"]
[UTCDate "2019.01.13"]
[UTCTime "04:15:39"]
[WhiteElo "1656"]
[BlackElo "1732"]
[WhiteRatingDiff "+13"]
[BlackRatingDiff "-13"]
[Variant "Standard"]
[TimeControl "900+15"]
[ECO "D00"]
[Termination "Normal"]
1. d4 d5 2. e3 Nf6 3. c4 c6 4. Qb3 Nbd7 5. Nc3 Nb6 6. cxd5 Nfxd5 7. Nxd5 cxd5 8. Bb5+ Bd7 9. Nf3 e6 10. Bxd7+ Qxd7 11. Ne5 Qc7 12. Bd2 Bd6 13. Rc1 Qe7 14. Qb5+ Kf8 15. f4 Bxe5 16. dxe5 Nc4 17. Bb4 1-0
It looks like the default format for this endpoint is application/x-chess-pgn
, however application/x-ndjson
is also available. (This from the API docs)
To tell the API you want application/x-ndjson
, you can use the Accept
header.
For example:
import requests, json
url = "https://www.lichess.org/api/games/user/mbellm"
r = requests.get(
url,
params={"max":2, "analysed":"true", "clocks":"true", "evals":"true", "opening":"true"},
headers={"Accept": "application/x-ndjson"}
)
r_text = r.content.decode("utf-8")
print(r_text)
Sample of r_text
:
{"id":"0ngCNyCN","rated":true,"variant":"standard","speed":"classical","perf":"classical","createdAt":1547495368806,"lastMoveAt":1547498443462,"status":"resign","players":{"white":{"user":{"id":"mbellm","name":"mbellm"},"rating":1715,"ratingDiff":-9,"analys
Edit
Since the response from this API is ndjson
or Newline Delimed JSON, calling json.loads
will not work properly on the response data. You can split the response data on the new line character "\n"
and then call json.loads
on each substring. Note that the response ends with "\n"
, so be sure not to call json.loads
on the final substring.
Here's an example:
games = [json.loads(s) for s in r_text.split("\n")[:-1]]