I have enabled logging in postgres. It logs in CSV. The ninth column is the query that's actually executed. I want to run a cut
command to pull that 9th column, but the issue is that the commas and newlines in the query would break the parser. Is there any way to change the delimiter and the newline character?
As suggested in PostgreSQL Manual you can use postgres to parse its own logs:
Here is simple bash script to automate the process.
#!/usr/bin/env bash
psql -t << EOF
CREATE TEMPORARY TABLE postgres_log
(
log_time timestamp(3) with time zone,
user_name text,
database_name text,
process_id integer,
connection_from text,
session_id text,
session_line_num bigint,
command_tag text,
session_start_time timestamp with time zone,
virtual_transaction_id text,
transaction_id bigint,
error_severity text,
sql_state_code text,
message text,
detail text,
hint text,
internal_query text,
internal_query_pos integer,
context text,
query text,
query_pos integer,
location text,
application_name text,
PRIMARY KEY (session_id, session_line_num)
);
COPY postgres_log FROM '$1' WITH csv;
SELECT query from postgres_log;
EOF
You can run it as: ./fetch-queries.sh /full/path/to/logfile.csv