presentation-memo.md in rabbit-slide-kenhys-debconf2020-online-2020.08.25.1 vs presentation-memo.md in rabbit-slide-kenhys-debconf2020-online-2020.08.25.2
- old
+ new
@@ -1,390 +1,322 @@
# Presentation memo
+Hi, thank you for coming to this session.
+
+I'll start the presentation "An experiment about personalized
+front-end of bugs.debian.org.
+
## NOTE
-This slide is published at Rabbit Slide Show
+This slide was already published at Rabbit Slide Show
+So, you can also download this presentation.
+
+## Personal profile
+
+Let me just start by introducing myself.
+
+I'm one of the Debian maintainers. and a fan of a Trackpoint and Wasa
+beef. I'm also working for ClearCode.
+
## ClearCode Inc.
-* My collegue gave me a feedback for this talk
+Usually, I'm developing and supporting software as an engineer.
+There is a culture to respect free software, my colleague gave me a
+feedback for this talk.
+
## Not talk about
-In this session, I won't talk about bugs.debian.org internal in details.
+In this session, this presentation is related to bugs.debian.org, but
+I will not talk about improving bugs.debian.org itself and internal
+details.
# Agenda
-* Why so curious about bugs.d.o?
-* The troublesome case about bugs.d.o
-* What is Fabre and why is it needed?
-* Demonstrate a front-end
-* Conclusion
+Here is the session agenda.
-# As a Maintainer
+First, I'll explain why so curious about bugs.d.o
+and show you the troublesome case.
-* Maintain some packages
- * <https://qa.debian.org/developer.php?login=kenhys>
- * groonga (Upstream releases monthly updates)
- * fcitx-imlist
- * libhinawa
- * ...
+Next, I'll explain how to solve this situation and demonstrate an
+actual front-end.
# We use bugs.d.o
-{:relative-height="100"}
+Here is the screenshot of bugs.d.o.
+I guess you are familiar with it.
+
# bugs.debian.org
-* Communicate each other by E-mail
-* Change the bug status by sending E-mail
- * Sending E-mail to control@bugs.d.o
- * Use control command in E-mail reply
+In bugs.d.o, we need to follow the rules.
+1. Communicate each other by E-mail
+2. Change the bug status by sending E-mail
+
# Why so curious about bugs.d.o?
-* I've received deprecated notification bugs
- * Bug# 956770 growl-for-linux: Depends on deprecated libappindicator
- * Bug# 955899 growl-for-linux: Depends on deprecated dbus-glib
+One day, I've received deprecated notification about growl-for-linux
+package.
+growl-for-linux is a notification software.
+
+One is related to libappindicator and the other is related to
+dbus-glib.
+
# How did you fixed?
-* Bug# 956770 growl-for-linux: Depends on deprecated libappindicator
- * <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956770>
- * Migrated to libayatana-indicator
- * It is easy because there are enough compatibility
- * Fixed pkg-config target and include path
+About libappindicator, just migrated to libayatana-indicator.
# How did you fixed? (again)
-* Bug# 955899 growl-for-linux: Depends on deprecated dbus-glib
- * <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=955899>
- * Migrated to GDBus
- * Use d-feet to detect interface change (Rhythmbox plugin)
- * Fixed a interface to follow GDBus API
- * Send PR to upstream
- * Fixed a potential Rhythmbox related bug, too
+About, dbus-glib, I need to fix changed interfaces and migrated into
+GDBus.
# growl-for-linux was fixed
-* But...What about other packages? 🤔
+Two bugs about growl-for-linux package was fixed, but I thought what
+about other packages?
# What about other packages?
-* libappindicator: deprecated in Debian; AppIndicator based applications, please switch to Ayatana (App)Indicator(s)
- * <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=895037>
-* libdbus-glib-1-dev: is deprecated
- * <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=895291>
+libappindicator and dbus-glib was tracked like this.
# libappindicator: deprecated in Debian;
-{:relative-height="80"}
+Against libappindicator, there are many bugs.
-* Many blocked bugs
-
# libdbus-glib-1-dev: is deprecated
-{:relative-height="80"}
+Against dbus-glib, there are too many blocked bugs.
-* *Too many* blocked bugs (Need to scroll)
-
-# Can you track blocked bugs?
-
-{:relative-height="80"}
-
-* Feel dizzy 😵
-
# How to track many blocked?
-* Use UserTags?
- * <https://wiki.debian.org/bugs.debian.org/usertags>
- * <https://udd.debian.org/cgi-bin/bts-usertags.cgi>
+In these case, we can use UserTags to track bugs.
# Does it work with libappindicator?
-{:relative-height="80"}
+For libappindicator, ayatana-appindicator is used as UserTag
-* UserTag is: *ayatana-appindicator*
-
# Does it work with libdbus-glib1-dev?
-* It is a case that UserTag is *not available*
-* It should be tagged...
+It should be tagged... but, UserTag is not available for dbus-glib.
# The troublesome case about bugs.d.o
-* UserTag is not always used
-* Many blocked bugs may be difficult to see
-* *How to solve this situation?*
+Like this, there is a case that UserTag is not always used. How to
+solve this situation?
# Use udd.debian.org!
-{:relative-width="80"}
+The answer is using udd.
+UDD can track usertags and blocking bugs.
- * Can track usertags
- * Can track blocking bugs
-
# In my experience, I need
-* Easy to:
+So far, I've fixed or sent feedback to some bugs, I became to feel the
+following ones are needed.
- * Track a specified bug (tagged/not tagged)
- * Find a bug that no one working on
- * Send control/submitter E-mail
- * Find affected bugs
+Tracking a specified bug
+Finding a bug that no one working on
+Sending control E-mail in required form
+Finding affected bugs
# What I need for?
-* Easy to:
+First topic is solved by using UDD.
- * ~~Track a specified bug (tagged/not tagged)~~ *Use UDD!*
- * Find a bug that no one working on 🤔
- * Send control/submitter E-mail 🤔
- * Find affected bugs 🤔
+But how about the rest issues?
-# Starting personal project
+# How to solve rest issues?
-* Concept
- * Make unstable life comfortable
- * Find a bug that no one working on
- * Easy to send control E-mail
- * Easy to find blocking bugs
- * Easy to find affected bugs
+Surely, It is a good manner to fix a bug, but it is not easy.
+It is also good to triage a bug, but it may be troublesome.
-# Starting personal project
+To know affected bugs are also important, but it is not so easy until
+it hits you.
-* Concept
- * Make unstable life comfortable
- * Find a bug that no one working on
- *Count comment in bug#NNNNNN*
- * Easy to send control E-mail
- *Add mailto: link explicitly*
- * Easy to find blocking bugs
- *List blocking bugs*
- * Easy to find affected bugs
- *Alert affected bugs*
+# Find affected bugs
-# What should I do?
+In the past, there are some grave/critical bugs It had affected boot
+sequence / logged in user session. I want to avoid these bugs.
-* Name the project 😀
-* Integrate with salsa.d.o account (optional)
-* Sync bug information
-* Import additional metadata
-* Make it accessible!
+# What about existing solution?
-# What should I do?
+I've searched existing useful services via Debian wiki.
-* *Name the project 😀*
- {::note}Naming is important!{:/note}
-* Integrate with salsa.d.o account (optional)
-* Sync bug information
-* Import additional metadata
-* Make it accessible!
+Developer Horizon seems well, but not available. Fabre also seems
+well, but discontinued already.
-# Named as Fabre
+I've also heard Debbugs Enhancement Suite. But it is a bit different
+what I want.
- {:relative-height="80"}
+# What about existing solution?
-cite: <https://en.wikipedia.org/wiki/Jean-Henri_Fabre>
+As a result, Debian Popularity Content and Public UDD Mirror seems
+useful.
-# Why Fabre?
+# Starting personal project
-* Reuse previous (maybe) similar project
- * <http://fabre.debian.net>
- * Around 2005, fulltext search service had been hosted
- * Undocumented Debian "debbugs internal" {::note}written in Japanese{:/note}
- * <https://tokyodebian-team.pages.debian.net/pdf2005/debianmeetingresume2005-fuyu.pdf>
+So I've started a personal project.
-# What should I do?
+The concept of this project is: "Make unstable life comfortable" It
+seems that it is ridiculous, but important for me.
-* Name the project 😀
- {::note}Naming is important!{:/note}
-* *Integrate with salsa.d.o account (optional)*
-* Sync bug information
-* Import additional metadata
-* Make it accessible!
+# Starting personal project
-# Integrate with salsa.d.o account (optional)
+To achieve this concept, I need to do some tasks.
-* Use GitLab(salsa.d.o) as an OAuth2 provider
- * <https://docs.gitlab.com/ee/api/oauth2.html>
-* No need to store sensitive password
+counting comment for each bugs,
+and adding separate mailto: link,
+checking installed packages to alert affected bugs.
-# What should I do?
+# How to realize concept
-* Name the project 😀
- {::note}Naming is important!{:/note}
-* Integrate with salsa.d.o account (optional)
-* *Sync bug information*
-* Import additional metadata
-* Make it accessible!
+Furthermore, collecting E-mail archives, popcon data, installed
+packages list and so on are also required.
-# Sync bug information
+# Collect bug information
-* Get E-mail archive with rsync
- * <https://salsa.debian.org/debbugs-team/debbugs>
- * rsync -av rsync://bugs-mirror.debian.org/bts-spool-db/
- {::note}It requires 15GiB!{:/note}
-* Use *.log and *.summary
+To collect E-mail archives, we can use bugs-mirror with rsync. E-mail
+archive contains *.log and *.summary.
# Extract .log and .summary
-* Use perl module which is used for bugs.d.o (Debbugs)
- * Debbugs::Log, Debbugs::Status, Debbugs::MIME and so on
- * *.log contains multiple raw E-mail content with control sequence
- * *.summary contains metadata of each bug
+There are many Perl modules to handle bugs.d.o related pieces of
+stuff.
# How to parse .log
- my $log = Debbugs::Log->new(log_name => $path);
- my @records = $log->read_all_records();
- for my $record (@records) {
- ...
- my $entity = Debbugs::MIME::parse_to_mime_entity($record);
- print "From: " . $entity->head->get("From");
- print "To: " . $entity->head->get("To");
- print "Subject: " . get_field($entity, "Subject");
- print "Date: " . $entity->head->get("Date");
- }
-{: lang="perl"}
+For example, Log and MIME module is used to parse .log file.
# How to parse .summary
- Format-Version: 3
- Submitter: Nis Martensen <nis.martensen@web.de>
- Subject: libexo-helpers: starting thunderbird fails for some mailto URIs
- Found-In: exo/0.12.4-1
- Tags: patch
- Package: libexo-helpers
- Message-Id: <159439678897.3833.10774950364480956704.reportbug@siamos.nes.net>
- Severity: normal
- Date: 1594396982
+Status module is used to parse .summary file.
+You can access these bugs's metadata.
- my $summary = Debbugs::Status::read_bug(summary => $path);
- print $summary->{"tags"}
-{: lang="perl"}
+# Collect Popcon data and so on
-# What should I do?
+As UDD provides public database access,
-* Name the project 😀
- {::note}Naming is important!{:/note}
-* Integrate with salsa.d.o account (optional)
-* Sync bug information
-* *Import additional metadata*
-* Make it accessible!
+So we can get popcon stats and investigate which bug affects unstable
+or not.
-# Import additional metadata
+# Collect installed packages information
-* UDD: UltimateDebianDatabase
- * <https://wiki.debian.org/UltimateDebianDatabase>
- * You can access with:
- psql --host=udd-mirror.debian.net --user=udd-mirror udd --password
- * Schema is available: <https://udd.debian.org/schema/udd.html>
- * Import popcon and package data
+As popularity-content command can collect installed binary packages
+information. It is useful to check recently used packages.
-# What should I do?
+# Process collected data and make it accessible!
-* Name the project 😀
- {::note}Naming is important!{:/note}
-* Integrate with salsa.d.o account (optional)
-* Sync bug information
-* Import additional metadata from UDD
-* *Make it accessible!*
+After collecting these information, updating bug information is
+required regularly.
+I've used debian-bugs-dist mailing list as a trigger to
+update and imported it into database.
+
# Prepared server specs for experiment
-* Start with small VPS instance (about 3.26 USD/mo)
- * 1vCPU
- * 1GiB memory
- * 20GiB disk space
- * Max 100Mbps bandwidth
+For my experiment, I've prepared small VPS instance.
-# Fabre internals
+# System diagrams
-{:relative-height="100"}
+Here is a simple diagram for explaining the system.
-# What is the trigger of syncing
+Web server is nginx, Application server is puma, and backend database is Groonga.
-* Subscribe debian-bugs-dist@lists.debian.org
-* Poll subscribed E-mail account
-* Fetch latest E-mail archives on bugs-mirror.debian.org
-* Convert & import E-mail archive data
+It access external services,
+E-mail server, UDD for metadata, Salsa for authentication.
-# Concept of Fabre
+# Named my project as Fabre
-* Easy to find a bug that no one working on
- * Count comment in bug#NNNNNN
-* Easy to send control E-mail
- * Add mailto: link explicitly
-* Easy to view blocking bugs
- * List blocking appropriately
+As diagram indicates,
+I've named this personal system as Fabre.
-# Easy to find a bug that no one working on
+# Why Fabre?
-{:relative-height="80"}
+Fabre is famous by the study of insects.
+I've borrowed from discontinued similar project in decades ago
-* Show each number of comments (yellow background color means no one working on yet)
+# Concept of Fabre (again)
-# Easy to send control E-mail
+The concept is this project is Make "unstable life" comfortable.
-{:relative-height="80"}
+# Finding a bug that no one working on
-* Show each mailto: button to control
+Here is the partial screenshot of Fabre.
+It shows each number of comments (orange background color means no one working on yet)
-# Easy to view blocking bugs
+# Sending control E-mail easily
-{:relative-height="80"}
+I've added each mailto: button to sending control mail
-* List blocking appropriately
+# Easy to view blocked bugs
-# What comes to next?
+I've aligned blocked bugs.
-* It looks useful, isn't it?
+# Finding affected important bugs
-# Personalize Fabre!
+I've added a dashboard to show fire icon. It indicates that there are affected bugs.
-* Add bookmark
-* Show affected bugs
-* Followup newer comments
+# How Fabre is good enough to work?
-# Personalize Fabre!
+I'll show you some demo about Fabre.
-* Add bookmark
- * *Link account and bug*
-* Show affected bugs
- * *Link account and installed packages list*
-* Followup newer comments
- * *Link account and search comment information*
+# Current status of Fabre
-# How to collect installed packages list?
+That's all about Fabre demonstration.
- POPULARITY-CONTEST-0 TIME:1596711793 ID:XXX ARCH:amd64 POPCONVER:1.70 VENDOR:Debian
- 1596672000 1570622400 libgail-common /usr/lib/x86_64-linux-gnu/gtk-2.0/modules/libgail.so
- 1596672000 1596585600 libavcodec58 /usr/lib/x86_64-linux-gnu/libavcodec.so.58.91.100 <RECENT-CTIME>
- 1596672000 1586736000 libgoa-1.0-0b /usr/lib/x86_64-linux-gnu/libgoa-1.0.so.0.0.0
- ...
- 1596672000 1586217600 libzvbi0 /usr/lib/x86_64-linux-gnu/libzvbi.so.0.13.2
- 1596672000 1590624000 udisks2 /usr/share/locale/ja/LC_MESSAGES/udisks2.mo
- END-POPULARITY-CONTEST-0 TIME:1596711817
+In this system, tracked bugs are reached to eighty-nine thousand.
+UDD tracks 90 thousand bugs, so it covers most of them.
-* Just upload popcon file
- * /usr/sbin/popularity-contest --su-nobody
+Currently, bug information is updated every 1 hour, and affected bugs are also updated every 1 day.
-# Personalized Fabre on dashboard
+Disk usage is about 1.5 GiB.
-{:relative-height="100"}
+# Weak point of Fabre
-# How Fabre is good enough to work?
+There is a one weak point about Fabre.
+out of memory is occurred sometimes.
-* Now ready to DEMO if there is a time to do
-
# Conclusion
-* In my experiment, mashup some data sources may improve UX
- * bug report
- * UDD metadata
- * installed packages list
-* There are plenty room of improvements to develop
+* There is plenty room of improvement to develop, so I'll go forward this experiment furthermore.
+* In my experiment, I've found that mashup some data sources may improve user experience.
# Any questions?
Could you speak slowly if you ask, please?
+
+# DEMO
+
+I'll demonstrate some features of Fabre.
+
+If you already sign-in with salsa account,
+You can also sign in with Fabre.
+
+You need explicitly allow to authorize with Fabre.
+
+After sign in, you can access the dashboard.
+
+In the dashboard, you can see a status.
+
+As you can see, there are three affected bugs,
+ten bookmarked bugs, and you installed thousand and six hundred packages.
+
+At the bottom of the dashboard, you can check the existence of newer comments since you commented finally.
+
+Let's see the affected bugs.
+These information is based on your installed packages.
+
+Let's see the bookmarked bugs.
+In every bug detail page, bookmark button is shown, so you can track bug freely.
+
+You can see recent update bugs, release critical bugs, and newcomer bugs.
+
+Here is the bug detail page,
+
+You can view original bug report via clicking link,
+tweet about bugs,
+and send control E-mail easily.
+