README.md in pgdexter-0.2.1 vs README.md in pgdexter-0.3.0

- old
+ new

@@ -10,12 +10,12 @@ First, install [HypoPG](https://github.com/dalibo/hypopg) on your database server. This doesn’t require a restart. ```sh cd /tmp -curl -L https://github.com/dalibo/hypopg/archive/1.0.0.tar.gz | tar xz -cd hypopg-1.0.0 +curl -L https://github.com/dalibo/hypopg/archive/1.1.0.tar.gz | tar xz +cd hypopg-1.1.0 make make install # may need sudo ``` > Note: If you have issues, make sure `postgresql-server-dev-*` is installed. @@ -43,27 +43,27 @@ ``` This finds slow queries and generates output like: ``` -2017-06-25T17:52:19+00:00 Started -2017-06-25T17:52:22+00:00 Processing 189 new query fingerprints -2017-06-25T17:52:22+00:00 Index found: genres_movies (genre_id) -2017-06-25T17:52:22+00:00 Index found: genres_movies (movie_id) -2017-06-25T17:52:22+00:00 Index found: movies (title) -2017-06-25T17:52:22+00:00 Index found: ratings (movie_id) -2017-06-25T17:52:22+00:00 Index found: ratings (rating) -2017-06-25T17:52:22+00:00 Index found: ratings (user_id) -2017-06-25T17:53:22+00:00 Processing 12 new query fingerprints +Started +Processing 189 new query fingerprints +Index found: public.genres_movies (genre_id) +Index found: public.genres_movies (movie_id) +Index found: public.movies (title) +Index found: public.ratings (movie_id) +Index found: public.ratings (rating) +Index found: public.ratings (user_id) +Processing 12 new query fingerprints ``` To be safe, Dexter will not create indexes unless you pass the `--create` flag. In this case, you’ll see: ``` -2017-06-25T17:52:22+00:00 Index found: ratings (user_id) -2017-06-25T17:52:22+00:00 Creating index: CREATE INDEX CONCURRENTLY ON "ratings" ("user_id") -2017-06-25T17:52:37+00:00 Index created: 15243 ms +Index found: public.ratings (user_id) +Creating index: CREATE INDEX CONCURRENTLY ON "public"."ratings" ("user_id") +Index created: 15243 ms ``` ## Connection Options Dexter supports the same connection options as psql. @@ -82,34 +82,62 @@ ``` host=localhost port=5432 dbname=mydb ``` -## Options +## Collecting Queries -Name | Description | Default ---- | --- | --- -exclude | prevent specific tables from being indexed | None -interval | time to wait between processing queries, in seconds | 60 -log-level | `debug` gives additional info for suggested indexes<br />`debug2` gives additional info for processed queries<br />`error` suppresses logging | info -log-sql | log SQL statements executed | false -min-time | only process queries consuming a min amount of DB time, in minutes | 0 +There are many ways to collect queries. For real-time indexing, pipe your logfile: -## Non-Streaming Modes +```sh +tail -F -n +1 <log-file> | dexter <connection-options> +``` -You can pass a single statement with: +Pass a single statement with: ```sh dexter <connection-options> -s "SELECT * FROM ..." ``` -or files with: +or pass files: ```sh dexter <connection-options> <file1> <file2> ``` +or use the [pg_stat_statements](https://www.postgresql.org/docs/current/static/pgstatstatements.html) extension: + +```sh +dexter <connection-options> --pg-stat-statements +``` + +### Collection Options + +To prevent one-off queries from being indexed, specify a minimum number of calls before a query is considered for indexing + +```sh +dexter --min-calls 100 +``` + +You can do the same for total time a query has run + +```sh +dexter --min-time 10 # minutes +``` + +Specify the format + +```sh +dexter --input-format csv +``` + +When steaming logs, specify the time to wait between processing queries + +```sh +dexter --interval 60 # seconds +``` + ## Examples Ubuntu with PostgreSQL 9.6 ```sh @@ -120,16 +148,61 @@ ```sh tail -F -n +1 /usr/local/var/postgres/server.log | dexter dbname ``` +## Tables + +You can exclude large or write-heavy tables from indexing with: + +```sh +dexter --exclude table1,table2 +``` + +Alternatively, you can specify which tables to index with: + +```sh +dexter --include table3,table4 +``` + +## Debugging + +See how Dexter is processing queries with: + +```sh +dexter --log-sql --log-level debug2 +``` + +## Analyze + +For best results, make sure your tables have been recently analyzed so statistics are up-to-date. You can ask Dexter to analyze tables it comes across that haven’t been analyzed in the past hour with: + +```sh +dexter --analyze +``` + ## Hosted Postgres Some hosted providers like Amazon RDS and Heroku do not support the HypoPG extension, which Dexter needs to run. See [how to use Dexter](guides/Hosted-Postgres.md) in these cases. ## Future Work [Here are some ideas](https://github.com/ankane/dexter/issues/1) + +## Upgrading + +Run: + +```sh +gem install pgdexter +``` + +To use master, run: + +```sh +gem install specific_install +gem specific_install https://github.com/ankane/dexter.git +``` ## Thanks This software wouldn’t be possible without [HypoPG](https://github.com/dalibo/hypopg), which allows you to create hypothetical indexes, and [pg_query](https://github.com/lfittl/pg_query), which allows you to parse and fingerprint queries. A big thanks to Dalibo and Lukas Fittl respectively.