interesting. the UID don't seem to be globally unique. two different
messages in 2 different mailboxes can have same UID


Mon Nov 22 23:12:49 EST 2010

- next, do window sensing and recreation. let user expand the message
  detail window and shrink it with some key like <cr>

I need status windows that show how many messagesin a box, when it was
last updated, etc

This app is coming along well.


Another reason to not allow a divergent offline store is that when the
schema changes on an update, it's easier to have the user rebuild the
locate version of the GMAIL 


Another anomaly. Threading seems to reverse the UIDs. 


Imap tasks should be batched so that we don't have the overhead of an
imap login each time.

The vertial 3 pan view should have a different set of columns in the
message list window.

Need command to reprocess text translation, also to display way eml.

NEXT. Learn how to do a popup window. Try out other Vim plugins! Do some
research online.

OR, do replies.

- work on email formatting

Flag new messages. And position cursor correctly on a refresh.




inputlist()


winrestcmd() potentially useful

1resize 20|vert 1resize 84|2resize 18|vert 2resize 84|3resize 39|vert 3resize 84|


------------------------------------------------------------------------
Thu Nov 25 16:55:07 EST 2010 {{{1

Trying new schema and more unix-inspired approach. First, the schema is
changed to allow unique messages to be shared across mailboxes via
message_refs. The Mailbox model has been removed.

Second, the tools should be smaller and less interdependent. There
should be a tool just for getting a text list of UID for a mailbox, and
then another that takes a UID and pulls down the raw eml, which I can
then use for testing.

The tool to insert into data should take two arguments, the mailbox name
and the UID, and the raw eml into STDIN.

Set up a test harness for all this email caching and data extraction
with rspec or test unit

------------------------------------------------------------------------
Fri Nov 26 12:08:26 EST 2010 {{{1

I need a text extract tool that works like iconv command line tool.

Lots of little tools for my workbench.


Now, I'm thinking of ditching the database. Use one document of
concatenated EML with message and mailbox info incorporated.


I tried using folds to present the email headers but I'm not sure it
works. Too many key strokes to unfold, fold, move to next message, and
unfold. Also, search doesn't work as expected since it searches through
all the text.

This suggests that using the database might be the best solution.


OK I've experimented and found a reasonably fast way to do imap searchs

e.g. 

  ruby lib/gmail.rb FROM monit BEFORE 8-Aug-2010 SINCE 1-Jun-2010

Now the question is how to get from output to load a specific message
and then possibly reply to it.

And later, we need to keep a contact list somehow. 

I also forgot how an email is sent with sendmail. Maybe with
authentication.

------------------------------------------------------------------------
Sun Nov 28 10:42:50 EST 2010

Got threaded message list fetch working

Marking messages for deletion or starring could be done very simply:
Just add a mark '*' or 'D' to the beginning of lines that should be
starred or deleted, then press a command key.

Just put one commented line at top of generated list. This is so the
viewer can refresh the list on demand when the user wants to. But the
focus should be on generating these imap search result lists from the
command line and saving the output to a file. Then the use can view the
file in the viewer. Should the file serve as a virtual mailbox?

User can alway do "search command | tee inbox" periodically and then
view the inbox in the viewer when a new message arrives.

There are even ways of setting up watches from the command to see if a
new message arrives in a certain mailbox. These recipes should be
written up in a tutorial on the website.


Wrote a wrapper startup script that you invoke like so

./wrapper inbox 20 all
./wrapper inbox 20 subject instantwatcher

This will generate the list file and open the viewer on it.

I think this is the workflow I want. It keeps the viewer lightweight and
prevents it from becoming a captive user interface.



Remember to convert uid to_i before calling IMAP methods.

Need to figure out a way to flag a line and show the change. Do we have
to use Vim data structures?

Also, be careful. The f command throws an error when there is no mailbox
and search term information at the top of the message list.


Make a view mode that lets one simple page through the messages and keep
track of which message is visible.

TODO Try d for trash and D for permanent Delete.


http://stackoverflow.com/questions/1430866/gmail-threading-imap-and-ruby

------------------------------------------------------------------------
Tue Nov 30 07:30:19 EST 2010

Start from command line, and then quit. Not meant to be an application
that you keep open. At least not for my uses.

But it would be better to start up the app in the Ruby wrapper

If it's contained in vim, I can use append(lnum, list) to insert
previous messages to the top of the buffer as the user scrolls back.

So two things to accomplish this morning: 
1. start the application in a Ruby wrapper
2. select mailbox from viewer
3. infinite paging back

Done:
1. Ruby wrapper
2. Update the current mailbox
3. Fast deletes, range flagging
4. slightly better multipart displaying

Won't do:
- infinite paging 
- starting mailbox from viewer

TODO
- send messages


Unicode korean characters are wider than ascii characters, so this will
mess up alignment, but I don't know what I can do about this.

------------------------------------------------------------------------
Wed Dec  1 15:39:39 EST 2010

- got composing messages working, switching between full screen message
  view and list view

TODO
- reply to messages

contacts
- import contacts by looking at sent mailbox. save contacts into a file
  and use that for autocompletion

split reply window veritcally: that's the way I wanted this interface

Quoting emails:

  http://en.wikipedia.org/wiki/Posting_style

stackoverflow:

  If you get the raw source of the messages you can relate messages to one
  another by looking at the following headers:

  Message-ID:
  <097819EBC7F79F4A850C8F088D35927302442A82AA@xxxxxxxxxxxxxxxxxxx>
  References: <4A8BE8F3.2060007@xxxxxxxxxxxxxx>
  In-Reply-To: <4A8BE8F3.2060007@xxxxxxxxxxxxxxx>
  link|flag
  answered Aug 28 '09 at 7:23
  Kees de Kooter
  1,7341512


TODO
- get full email addresses including names
- fix reply to quote header
- address book
- later: 
  - append to text file
  - display and save attachments
Thurs
  - start packaging
- need a better way to refresh a stale IMAP connection
  - try wrapping client calls with Timeout. Then call daemon to refresh
    connection if things take too long.

* Need to save mail list buffer or else it forgets what user has flagged
or deleted.


- will have to write out message list buffer

- IMAP fails from parsing email from 

  D, [2010-12-02T04:14:47.034816 #26464] DEBUG -- : error fetching uid
  83682
  D, [2010-12-02T04:14:47.034884 #26464] DEBUG -- : unknown token -
  "\"[Christian" (Net::IMAP::ResponseParseError)


It is vital to rewrite the message list buffer to file. This is a
significant bug. Since new messages gotten via update disappear when
coming back from full screen message view.

Also, let use go forward and back while reading full messages.
Or even in split view. Don't bind n since we need it for search. Use
CTRL-n, CTRL-p? or leader n leader 

Need better recovery from expired IMAP session. At least some kind of
warning so that use retries the previous action that failed.

TODO
- gmail charset correction


------------------------------------------------------------------------
Tue Dec  7 09:36:01 EST 2010

Instead of autocomplete mailbox chooser, have a generic search command
line buffer that might autocomplete the mailbox for you. 

This simplifies the user interface and makes it more coherent.

You will always specify a mailbox and a command. Get the user used to
doing searches like this.

I'm also thinking of keep the viewer thin and not that featureful.
I'll emphasize the command line features, like using search criteria to
pipe emails into a text file.

Change the architecture so that the daemon is fired up in a separate
process.

I don't think I need to download the RFC822 header, the envelope has
this data. 

Done. 

Next, in the search window, I can keep a history of previous searches.
And just execute the command that the user hits carriage return on.

    ENVELOPE: !ruby/struct:Net::IMAP::Envelope
      date: Tue, 7 Dec 2010 11:25:47 -0500
      subject: "Error message: /titles.new"
      from:
      - !ruby/struct:Net::IMAP::Address
        name:
        route:
        mailbox: error
        host: instantwatcher.com
      sender:
      - !ruby/struct:Net::IMAP::Address
        name:
        route:
        mailbox: error
        host: instantwatcher.com
      reply_to:
      - !ruby/struct:Net::IMAP::Address
        name:
        route:
        mailbox: error
        host: instantwatcher.com
      to:
      - !ruby/struct:Net::IMAP::Address
        name:
        route:
        mailbox: dhchoi
        host: gmail.com
      cc:
      bcc:
      in_reply_to:
      message_id: <4cfe600b1cd1f_202540e109c8ee@ubuntu.tmail>




(druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:3235:in `parse_error': unknown token - "\"[Christian" (Net::IMAP::ResponseParseError)
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:3180:in `next_token'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:3097:in `lookahead'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2995:in `nstring'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2183:in `envelope'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2170:in `envelope_data'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2145:in `msg_att'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2125:in `numeric_response'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2067:in `response_untagged'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:2047:in `response'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1973:in `parse'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1124:in `get_response'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1036:in `receive_responses'
        from (druby://127.0.0.1:61676) /Users/choi/.rvm/rubies/ruby-1.9.2-p0/lib/ruby/1.9.1/net/imap.rb:1023:in `block in initialize'
        from run.rb:8:in `block in <main>'
        from run.rb:7:in `open'
        from run.rb:7:in `<main>'



    # Sends a APPEND command to append the +message+ to the end of
    # the +mailbox+. The optional +flags+ argument is an array of 
    # flags to initially passing to the new message.  The optional
    # +date_time+ argument specifies the creation time to assign to the 
    # new message; it defaults to the current time.
    # For example:
    #
    #   imap.append("inbox", <<EOF.gsub(/\n/, "\r\n"), [:Seen], Time.now)
    #   Subject: hello
    #   From: shugo@ruby-lang.org
    #   To: shugo@ruby-lang.org
    #   
    #   hello world
    #   EOF
    #
    # A Net::IMAP::NoResponseError is raised if the mailbox does
    # not exist (it is not created automatically), or if the flags,
    # date_time, or message arguments contain errors.




Next, I think I need to restore the vertical mailbox selection mode, and
then have the search window execute in the current mailbox.

Finally, I need to eliminate the limit number (on search) and just to dynamic
back-paging when a user gets to the top line, which will say "Get 40
more messages" or something.

Also, forget 2 pane, just toggle full screen message and list view

Finally, I need to do autocompletion for contacts.



done today:
- backpaging
- single window at a time
- vertical mailbox selection
- no more yaml

OK next
- autocompletion for contacts
- extract and show attachments, bytesize
  - can't show this without downloading RFC822.header
- autocomplete better for mailbox
- STTY adjustment `stty size` from the vim client and sent to back end?

This url doesn't open 

  http://twitter.com/#!/kanter/status/12398658028834816

  Because of the # or !. Wrapping it in quotes double or singles
  doesn't fix the prob. Need to use system and shellescape

TODO
- display current mailbox somewhere, and also search query
- allow splitting the window to keep multiple messages open. Don't call only. 
- move to - done
- contacts!
- character encodings

echo winwidth(1) 
  use this instead of system('stty size') to get window width
  - then adjust the column sizes

:set statusline=%<%f\ %h%m%r%=%-14.(%l,%c%V%)\ %P

  - in .vmail/ dir?

------------------------------------------------------------------------
Thu Dec  9 10:13:53 EST 2010

- got status line to show current mailbox. Next I just need to show the
  current search query!
- got bells to go away in most cases
- need to work on autocomplete for addressbook
- getting there!

- ok i got statusline working in principle

- next, do contacts
- fix encoding issues
- get ready to release. Monday seems doable.

for contacts, use dictionary file option in vim


DONE
- contacts, proof of concept
- encoding issues (some at least)
- get reply template recipients right
- forwarding (not with attachments)
- auto-adjusting column widths depending on window width
- for older message, replace time with year
- signature
- open hyperlink in browser (OS X only?)
- open html in browser
- when show_next_message, get message without flashing the list
  - solved by restoring two pane view
- when loading update, position cursor on first new message
- spacebar in message window to fullscreen the message and toggle back
- spam range
- in list, show message sizes 
- prevent the "8 more lines" and wait for enter. 2 is the threshhold
- fix compose message, reply windowing
- mapping improvements:
  - straight to reply from list window
- fixed addresses in recipients to a reply all 
- position cursor conveniently on opening compose windows
- show num additional when in sent msg list
- save drafts on server to [Gmail]/Drafts
- use unquote_and_convert for subject lines?
- attachments extraction
- encoding issues (korean) 
  - plus use iconv for 1.8
  - search for from kr
  - can't rely on mail.encoding to be non-nil
- if html version, command to open
- create wrapper exec to start both daemon and client from same
  window. redirect streams from daemon to a log file
- package gem
  - vmail (client)
- mvim = allow using through VMAIL_EDITOR env variable
- need to put mailbox and search on status line from startup
- c-j c-k to go to prev/next message
- turn search into an input() style prompt
- remember and show last search
- map u update from message window      
- do search after loading vim or mvim and send size first
- vim resize event

next:
- sending an attachment
  - can specify a directory, then all the items in the directory get
    attached and sent
- forwarding attachments
  - could be "attach: [uid] attachments to signal"
- put instructions in message window; put in a top section 
  - help /readme
  - help document should just be readme? or part of it.
- follow mysql and use -u and -p flags on startup of server?
  - omitting -p flag forces prompt
  - get rid of config file
- use temp files
  - tempname()
  - system() allows a parameters that is written to tmp file and passed
    to stdin
- print all selected messages into file (append)
  - can pipe through a command line tool, so we don't need to clutter
    viewer.vim with more mappings and functions
- ,s s confusion? Star vs search
- enhance contacts auto fix with more advanced vim script
  - sort contacts by frequency, then take first 10 or so of any match
- mvim - window width not correct
- mvim - starred messages not syntax colored

later:
- mvim redrawstatus line bug
  http://vim.1045645.n5.nabble.com/Redrawing-bug-in-MacVim-Command-T-since-commit-ba44868-td3248742.html
- allow one daemon, multiple clients (select mailbox?)
- remember searches?
- archive function (shortcut for move to all mail)
- crate to package app as a OS X app?
  http://rubyconf2008.confreaks.com/crate-packaging-your-ruby-application.html
  - need to bundle macvim
- if move to unknown mailbox, create the mailbox, then do the move
- some lingering encoding issues
- something is wrong with cursorline sometimes
- try flashing progress in echo line
  - straight to switch mailbox, etc from message window
- allow deliver command from any buffer or window, as long as headers are at top
- range selection from : prompt
- starring 
  - do fast action like deletes
- reload after window resize
- sometimes update doesn't work - bug
- show total messages from a search, showing 100
- message threads



------------------------------------------------------------------------

Threading
how to find the message that a message is in reply-to

imap.search ["HEADER", "MESSAGE-ID", "<757FC46C-9394-494E-91B4-B051F48419DA@prx.org>"]

:! ls %

  % is filename