README.md in pgdexter-0.2.1 vs README.md in pgdexter-0.3.0
- old
+ new
@@ -10,12 +10,12 @@
First, install [HypoPG](https://github.com/dalibo/hypopg) on your database server. This doesn’t require a restart.
```sh
cd /tmp
-curl -L https://github.com/dalibo/hypopg/archive/1.0.0.tar.gz | tar xz
-cd hypopg-1.0.0
+curl -L https://github.com/dalibo/hypopg/archive/1.1.0.tar.gz | tar xz
+cd hypopg-1.1.0
make
make install # may need sudo
```
> Note: If you have issues, make sure `postgresql-server-dev-*` is installed.
@@ -43,27 +43,27 @@
```
This finds slow queries and generates output like:
```
-2017-06-25T17:52:19+00:00 Started
-2017-06-25T17:52:22+00:00 Processing 189 new query fingerprints
-2017-06-25T17:52:22+00:00 Index found: genres_movies (genre_id)
-2017-06-25T17:52:22+00:00 Index found: genres_movies (movie_id)
-2017-06-25T17:52:22+00:00 Index found: movies (title)
-2017-06-25T17:52:22+00:00 Index found: ratings (movie_id)
-2017-06-25T17:52:22+00:00 Index found: ratings (rating)
-2017-06-25T17:52:22+00:00 Index found: ratings (user_id)
-2017-06-25T17:53:22+00:00 Processing 12 new query fingerprints
+Started
+Processing 189 new query fingerprints
+Index found: public.genres_movies (genre_id)
+Index found: public.genres_movies (movie_id)
+Index found: public.movies (title)
+Index found: public.ratings (movie_id)
+Index found: public.ratings (rating)
+Index found: public.ratings (user_id)
+Processing 12 new query fingerprints
```
To be safe, Dexter will not create indexes unless you pass the `--create` flag. In this case, you’ll see:
```
-2017-06-25T17:52:22+00:00 Index found: ratings (user_id)
-2017-06-25T17:52:22+00:00 Creating index: CREATE INDEX CONCURRENTLY ON "ratings" ("user_id")
-2017-06-25T17:52:37+00:00 Index created: 15243 ms
+Index found: public.ratings (user_id)
+Creating index: CREATE INDEX CONCURRENTLY ON "public"."ratings" ("user_id")
+Index created: 15243 ms
```
## Connection Options
Dexter supports the same connection options as psql.
@@ -82,34 +82,62 @@
```
host=localhost port=5432 dbname=mydb
```
-## Options
+## Collecting Queries
-Name | Description | Default
---- | --- | ---
-exclude | prevent specific tables from being indexed | None
-interval | time to wait between processing queries, in seconds | 60
-log-level | `debug` gives additional info for suggested indexes<br />`debug2` gives additional info for processed queries<br />`error` suppresses logging | info
-log-sql | log SQL statements executed | false
-min-time | only process queries consuming a min amount of DB time, in minutes | 0
+There are many ways to collect queries. For real-time indexing, pipe your logfile:
-## Non-Streaming Modes
+```sh
+tail -F -n +1 <log-file> | dexter <connection-options>
+```
-You can pass a single statement with:
+Pass a single statement with:
```sh
dexter <connection-options> -s "SELECT * FROM ..."
```
-or files with:
+or pass files:
```sh
dexter <connection-options> <file1> <file2>
```
+or use the [pg_stat_statements](https://www.postgresql.org/docs/current/static/pgstatstatements.html) extension:
+
+```sh
+dexter <connection-options> --pg-stat-statements
+```
+
+### Collection Options
+
+To prevent one-off queries from being indexed, specify a minimum number of calls before a query is considered for indexing
+
+```sh
+dexter --min-calls 100
+```
+
+You can do the same for total time a query has run
+
+```sh
+dexter --min-time 10 # minutes
+```
+
+Specify the format
+
+```sh
+dexter --input-format csv
+```
+
+When steaming logs, specify the time to wait between processing queries
+
+```sh
+dexter --interval 60 # seconds
+```
+
## Examples
Ubuntu with PostgreSQL 9.6
```sh
@@ -120,16 +148,61 @@
```sh
tail -F -n +1 /usr/local/var/postgres/server.log | dexter dbname
```
+## Tables
+
+You can exclude large or write-heavy tables from indexing with:
+
+```sh
+dexter --exclude table1,table2
+```
+
+Alternatively, you can specify which tables to index with:
+
+```sh
+dexter --include table3,table4
+```
+
+## Debugging
+
+See how Dexter is processing queries with:
+
+```sh
+dexter --log-sql --log-level debug2
+```
+
+## Analyze
+
+For best results, make sure your tables have been recently analyzed so statistics are up-to-date. You can ask Dexter to analyze tables it comes across that haven’t been analyzed in the past hour with:
+
+```sh
+dexter --analyze
+```
+
## Hosted Postgres
Some hosted providers like Amazon RDS and Heroku do not support the HypoPG extension, which Dexter needs to run. See [how to use Dexter](guides/Hosted-Postgres.md) in these cases.
## Future Work
[Here are some ideas](https://github.com/ankane/dexter/issues/1)
+
+## Upgrading
+
+Run:
+
+```sh
+gem install pgdexter
+```
+
+To use master, run:
+
+```sh
+gem install specific_install
+gem specific_install https://github.com/ankane/dexter.git
+```
## Thanks
This software wouldn’t be possible without [HypoPG](https://github.com/dalibo/hypopg), which allows you to create hypothetical indexes, and [pg_query](https://github.com/lfittl/pg_query), which allows you to parse and fingerprint queries. A big thanks to Dalibo and Lukas Fittl respectively.