# Changelog

## [ToDo]
`reek lib/markdown_exec.rb --config .reek`

- user settings
- hidden w , w/o () in names
- fix regexp in pathnames

- tree display

- [ ] mde options, user prompt, in md file or included file

- [ ] include blocks from local md file

- execute? yes/no/save/clipboard + record/edit/history

- [ ] list, view saved output

- completion 
    - [ ] include blocks in md file

- [ ] ruby gem data model

- [ ] extract yaml block into stdout
- [ ] extract json block into stdout

- [ ] import yaml, json data into environment

- [ ] yq filter with imported or named yaml, json data

- [ ] yaml dump of options w/ detail

- [ ] re-exec last script v re-run named block in last script
- [ ] repeat to reload last doc and block
- [ ] option to log blended, timeline; stdin, stdout, stderr; labels: prefix and blocks
- [ ] ren logged_stdout_filename_prefix to saved_stdout_filename_prefix
- [ ] ignore '#' in fenced code blocks

- [ ] write named block, can be included
- [ ] file type per block type
- [ ] file name per block name or specified in quotes ("")
- [ ] overwrite-rules for writing blocks
- [ ] erase-rules for written blocks
- [ ] files to create named at top of script to execute; written prior to start of script

- [ ] tab completion example ascii demo

- [ ] parameters or (env vars) in menu
- [ ] config menu or read env vars
- [ ] enable/disable script, output saving per file
- [ ] keep values between runs so same env vars are not prompted

- [ ] option to list full menu

- [ ] task confirmation block option or bash template, env names

- [ ] configuration block `mde_config` anywhere in file
- [ ] configuration block `mde_config` anywhere in folder config file
      ```yaml :(mde_config)```
- [x] fix execution stdin, stdout to allow for ask function
- [ ] fix execution stdin, stdout to allow for tty-prompt
- [ ] accept stdin as filename `-`
- [ ] fix
        $ bin/mde
        Choose a file: * Exit
        No blocks found.
        Choose a block: * Exit

- [ ] accept `!` shell execute command at prompt
- [ ] menu response: re-exec last saved script (like mde --select-recent-script)

- [ ] process shebang in markdown
- [ ] ignore shebang in markdown in rest of processing
- [! ] search for document in path
- [! ] search for document in custom path
- [ ] doc use of `#!/usr/bin/env mde`
- [ ] command to export each command to file, menu for list

- [ ] option blocks to exclude by shell

- #+BEGIN_SRC sh :results silent
  #+END_SRC

- [ ] requires extra CRs after ctrl-C of earlier script?

- [! ] silent for `mde file block -q 0`
- [! ] define for block: env var name, prompt if missing, default value
- [! ] display env vars in menu
- [! ] mde md -- --arg1 --arg2

- [! ] display tasks (regex) similar to current `:::`
  diff done, tbd, important, progress, note, timestamp

- [! ] redo block selection logic in
    def self.fcb_select?(options, fcb)
    used for selecting for 1) menu, 2) script composition, 3) list?
    layers 1) block name in cli, 2) required blocks, 3) hidden, 4) selected, 5) excluded, 6) also per shell name
- [ ] opt to add block name as comment in bash script
- [ ] marker in file separating VARS from CODE in bash script
  [ ] VARS, CODE section separately sourceable/executable
- [ ] config to hide unnamed bash blocks
- [ ] default to No after Save of script

- [ ] block into file; template

- [! ] option to use most recent named block if dupiclate, currently appends same-name blocks but lists twice in menu

- [! ] improve error when imported file is not found

- [ ] decorations for import block
prefix line(s)
inline replacements
suffix line(s)

```ruby
##
# Replace substrings in an input string based on a regular expression pattern
# with named capture groups. The replacements are formatted using a provided
# format string. Additional context can be provided to supplement or override 
# the named captures in the format string.
#
# @param input_str [String] The input string to process.
# @param regex [Regexp] The regular expression pattern with named capture groups.
# @param format_str [String] The format string for sprintf.
# @param context [Hash] Additional context to supplement or override named captures.
#
# @return [String] The processed string after replacements.
#
# def gsub_format(input_str, regex, format_str, context: {})
#   input_str.gsub(regex) do
#     captures_hash = $~.names.each_with_object({}) do |name, hash|
#       hash[name.to_sym] = $~[name]
#     end

#     ### add import file name, line number, line, to captures_hash, chain
#     # $~ (MatchData) - MatchData object created from the match; thread-local and frame-local. - English - $LAST_MATCH_INFO.
#     # $& (Matched Substring) - The matched string. - English - $MATCH.
#     # $` (Pre-Match Substring) - The string to the left of the match. - English - $PREMATCH.
#     # $' (Post-Match Substring) - The string to the right of the match. - English - $POSTMATCH.
#     # $+ (Last Matched Group) - The last group matched. - English - $LAST_PAREN_MATCH.

#     sprintf(format_str, context.merge(captures_hash))
#   end
# end

class Regexp
  def gsub_format(input_str, format_str, context: {})
    input_str.gsub(self) do
      captures_hash = $~.names.each_with_object({}) do |name, hash|
        hash[name.to_sym] = $~[name]
      end

#     ### add import file name, line number, line, to captures_hash, chain
#     # $~ (MatchData) - MatchData object created from the match; thread-local and frame-local. - English - $LAST_MATCH_INFO.
#     # $& (Matched Substring) - The matched string. - English - $MATCH.
#     # $` (Pre-Match Substring) - The string to the left of the match. - English - $PREMATCH.
#     # $' (Post-Match Substring) - The string to the right of the match. - English - $POSTMATCH.
#     # $+ (Last Matched Group) - The last group matched. - English - $LAST_PAREN_MATCH.

      # # Add file name, line number, line to captures_hash
      # captures_hash[:file_name] = $~.pre_match.split("\n").last
      # captures_hash[:line_number] = $~.pre_match.count("\n") + 1
      # captures_hash[:line] = $&

      sprintf(format_str, context.merge(captures_hash))
    end
  end
end

# # Example usage:
# str = "123 example"
# re = /(?<foo>\d+) (?<bar>\w+)/
# fmt = "%<foo>d : %<bar>s"
# new_str = gsub_format(str, re, fmt)
# puts new_str  # Outputs: 123 : example

require 'minitest/autorun'
require_relative 'path_to_your_file' # Make sure to replace this with the path to the file containing the function

class ReplaceWithFormatTest < Minitest::Test
  def test_basic_replacement
    input_str = "123 example"
    re = /(?<foo>\d+) (?<bar>\w+)/
    fmt = "%<foo>d : %<bar>s"
    
    result = gsub_format(input_str, re, fmt)
    
    assert_equal "123 : example", result
  end
  
  def test_no_match
    input_str = "This is a test."
    re = /(?<foo>\d+) (?<bar>\w+)/
    fmt = "%<foo>d : %<bar>s"
    
    result = gsub_format(input_str, re, fmt)
    
    assert_equal "This is a test.", result
  end
  
  def test_multiple_matches
    input_str = "123 example, 456 test"
    re = /(?<foo>\d+) (?<bar>\w+)/
    fmt = "[%<foo>d %<bar>s]"
    
    result = gsub_format(input_str, re, fmt)
    
    assert_equal "[123 example], [456 test]", result
  end
  
  def test_different_named_captures
    input_str = "Jane is 25 years old."
    re = /(?<name>\w+) is (?<age>\d+)/
    fmt = "%<name>s's age is %<age>d"
    
    result = gsub_format(input_str, re, fmt)
    
    assert_equal "Jane's age is 25", result
  end
  
  def test_with_context
    input_str = "Jane is 25 years old."
    re = /(?<name>\w+) is (?<age>\d+)/
    fmt = "%<name>s's age is %<age>d and she lives in %<city>s"
    
    result = gsub_format(input_str, re, fmt, context: { city: "New York" })
    
    assert_equal "Jane's age is 25 and she lives in New York", result
  end
end

require 'minitest/autorun'
require_relative 'path_to_your_file' # Ensure this path is correct

class RegexpGsubFormatTest < Minitest::Test
  def test_basic_replacement
    input_str = "123 example"
    re = /(?<foo>\d+) (?<bar>\w+)/
    fmt = "%<foo>d : %<bar>s"
    
    result = re.gsub_format(input_str, fmt)
    
    assert_equal "123 : example", result
  end

  # ... [other tests remain mostly unchanged, just updating the method call]

  def test_with_context
    input_str = "Jane is 25 years old."
    re = /(?<name>\w+) is (?<age>\d+)/
    fmt = "%<name>s's age is %<age>d and she lives in %<city>s"
    
    result = re.gsub_format(input_str, fmt, context: { city: "New York" })
    
    assert_equal "Jane's age is 25 and she lives in New York", result
  end
end

```

## [1.3.5] - 2023-10-05

### Changed

- Fix display of menu dividers.

## [1.3.3] - 2023-10-03

### Added

- Nest scripts by using an `import` directive.

### Changed

- Convert constants for block selection into options.

## [1.3.2] - 2022-11-12

### Added

- Add RSpec tests for internal classes

## [1.3.1] - 2022-10-29

### Added

- Delay to allow all command output to be received
- Display an error message when the specified document file is missing
- Options to display, format and colorize menu dividers and demarcations
- Tab completion for short option names

### Changed

- Fix handling of document supplied by process substitution

## [1.3.0] - 2022-07-16

### Added

- Short name `-p` for `--user-must-approve` option
  Enable/disable pause for user to review and approve script
- Automatic wrapping for data in blocks of yaml data eg ` ```yaml `
  Data is written to the file named in the fenced block heading
- Data transformations are embedded in the script at every invocation
  with arguments to the transformation as stdin and stdout for the `yq` process
  eg `export fruit_summary=$(yq e '[.fruit.name,.fruit.price]' fruit.yml)`
  for invocation `%(summarize_fruits <fruit.yml >fruit_summary)`
  and transformation `[.fruit.name,.fruit.price]`
- Option to extract document text and display it as disabled items in-line with the blocks in the selection menu.
  Add options for constants used in parsing.
- [x] yaml processing
    - ```yaml :(make_fruit_file) >fruit.yml```
      write to: fruit.yml
    - ```yq [summarize_fruits] +(make_fruit_file) <fruit.yml =color_price```
      not included in linear script
      read from: fruit.yml
      result into var: color_price instead of stdout
    - ```bash :show_fruit_yml +(summarize_fruits)```
      include summarize_fruits
      output value of var color_price

### Changed

- Refactoring
    - Run-time menu in YAML file.
    - Tap module initialization

## [1.2.0] - 2022-06-11

### Added

- Options
    - Display document name in block selection menu
    - Display headings (levels 1,2,3) in block selection menu
- Trap Ctrl-C (SIGTERM) while script is executing.
  Completes MDE processes such as saving output and reporting results.

### Changed

- Refactoring

## [1.1.1] - 2022-05-25

### Added

- Post-install instructions to add tab completions permanently to the shell.

### Changed

- Improve handling of threads ending while executing scripts.

## [1.1.0] - 2022-05-21

### Added

- Display administrative output (command, save files) for executed blocks.
- Select base, administrative output as hierarchical output (MDE_DISPLAY_LEVEL).
- The user may select Exit, the first option, to quit the program.
- The block-selection menu is re-displayed after an approved script is exectued.
- Pause for and pass through standard input in scripts executed by the tool.
- Options
    - chmod for saved scripts
    - shebang for saved scripts
    - shell for executed and saved scripts

### Changed

- Exit option is at top of each menu.
- Single-stage tab completion, default
    - Presents matching options when current word starts with `-`
    - Presents directories and files otherwise.
- Two-stage tab completion for option arguments.
    - When prior word is an option and current word is empty
    - Presents option type on first tab, eg `.BOOL.` for a boolean option.
    - Presents option default value on second tab, eg `0` for false.
- Write STDOUT, STDERR, STDIN in sections to saved output file.

## [1.0.0] - 2022-04-26

### Added

- Command `--pwd` to print the gem's home folder.
- Command `--select-recent-output` to select and open a recent output log file.
e.g. `MDE_OUTPUT_VIEWER_OPTIONS="-a '/Applications/Sublime Text.app'" mde --select-recent-output`
- Command `--tab-completions` to list the application options.
- Tab completion script for Bash shell.

### Changed

- File names for saved scripts.
- Hide blocks with empty names, e.g. `:()`.

## [0.2.6] - 2022-04-07

### Changed

- Default values for command line options.

## [0.2.5] - 2022-04-03

### Added

- Command `--list-default-env` to show default configuration as environment variables.
- Command `--list-default-yaml` to show default configuration as YAML.
- Option to exit program when selecting files or blocks.

### Changed

- Composition of menu to facilitate reports.
- List default values in menu help.

## [0.2.4] - 2022-04-01

### Added

- Command `--list-recent-scripts` to list the last *N* saved scripts.
- Command `--run-last-script` to re-run the last saved script.
- Command `--select-recent-script` to select and execute a recently saved script.

| YAML Name | Environment Variable | Option Name | Default | Purpose |
| :--- | :--- | :--- | :--- | :--- |
| list_count | MDE_LIST_COUNT | `--list-count` | `16` | Max. items to return in list |
| logged_stdout_filename_prefix | MDE_LOGGED_STDOUT_FILENAME_PREFIX | | `mde` | Name prefix for stdout files |
| save_execution_output | MDE_SAVE_EXECUTION_OUTPUT | `--save-execution-output` | False | Save standard output of the executed script |
| saved_script_filename_prefix | MDE_SAVED_SCRIPT_FILENAME_PREFIX | | `mde` | Name prefix for saved scripts |
| saved_script_folder | MDE_SAVED_SCRIPT_FOLDER | `--saved-script-folder` | `logs` | Saved script folder |
| saved_script_glob | MDE_SAVED_SCRIPT_GLOB | | `mde_*.sh` | Glob matching saved scripts |
| saved_stdout_folder | MDE_SAVED_STDOUT_FOLDER | `--saved-stdout-folder` | `logs` | Saved stdout folder |

### Changed

- Fix saving of executed script.
- Sort configuration keys output by `-0` (Show configuration.)

## [0.2.3] - 2022-03-29

### Added

- `rubocop` checks.
- Added optional command-line positional arguments:
  1. document file
  2. block name

	e.g. `mde doc1 block1` will execute the block named `block1` in file `doc1`.

### Changed

- Naming saved script files: The file name contains the time stamp, document name, and block name.
- Renamed folder with fixtures.
- Command options:

| YAML Name | Environment Variable | Option Name | Default | Purpose |
| :--- | :--- | :--- | :--- | :--- |
| debug | MDE_DEBUG | `--debug` | False | Output debugging information (verbose) |
| filename | MDE_FILENAME | `--filename` | `README.md`* | File name of document |
| output_execution_summary | MDE_OUTPUT_STDOUT_SUMMARY | `--output-execution-summary` | False | Display summary for execution |
| output_script | MDE_OUTPUT_SCRIPT | `--output-script` | False | Display script |
| output_stdout | MDE_OUTPUT_EXECUTION | `--output-stdout` | True | Display standard output from execution |
| path | MDE_PATH | `--path` | `.` | Path to document(s) |
| save_executed_script | MDE_SAVE_EXECUTED_SCRIPT | `--save-executed-script` | False | Save executed script |
| saved_script_folder | MDE_SAVED_SCRIPT_FOLDER | `--saved-script-folder` | `logs` | Saved script folder |
| select_page_height | MDE_SELECT_PAGE_HEIGHT |  | `12` | Menu page height (maximum) |
| user_must_approve | MDE_USER_MUST_APPROVE | `--user-must-approve` | True | Pause to approve execution |

- Configuration options:

| YAML Name | Environment Variable | Default |
| :--- | :--- | :--- |
| block_name_hidden_match | MDE_BLOCK_NAME_HIDDEN_MATCH  | `^\(.+\)$` |
| block_name_match | MDE_BLOCK_NAME_MATCH  | `:(?<title>\S+)( \|$)` |
| block_required_scan | MDE_BLOCK_REQUIRED_SCAN  | `\+\S+` |
| fenced_start_and_end_match | MDE_FENCED_START_AND_END_MATCH  | ``^`{3,}`` |
| fenced_start_ex_match | MDE_FENCED_START_EX_MATCH  | ``^`{3,}(?<shell>[^`\s]*) *(?<name>.*)$`` |
| heading1_match | MDE_HEADING1_MATCH  | `^# *(?<name>[^#]*?) *$` |
| heading2_match | MDE_HEADING2_MATCH  | `^## *(?<name>[^#]*?) *$` |
| heading3_match | MDE_HEADING3_MATCH  | `^### *(?<name>.+?) *$` |
| md_filename_glob | MDE_MD_FILENAME_GLOB  | `*.[Mm][Dd]` |
| md_filename_match | MDE_MD_FILENAME_MATCH  | `.+\\.md` |

Most options can be configured in multiple ways. In order of use (earliest superceded by last):
1. environment variables
2. the configuration file `.mde.yml` in the current folder
3. command line arguments

#### Representing boolean values

Boolean values for options are specified as strings and interpreted as:
| Value | Boolean |
| :---: | :---: |
| *empty string* | False |
| `0` | False |
| `1` | True |
| *anything else* | True |

E.g. `opt1=1` will set option `opt1` to True.

Boolean options configured with environment variables:
- Set to `1` or non-empty value to save executed scripts; empty or `0` to disable saving.
  e.g. `export MDE_SAVE_EXECUTED_SCRIPT=1`
  e.g. `export MDE_SAVE_EXECUTED_SCRIPT=`
- Specify variable on command line.
  e.g. `MDE_SAVE_EXECUTED_SCRIPT=1 mde`

## [0.2.2] - 2022-03-17

- Update documentation.

## [0.2.1] - 2022-03-12

- Accept file or folder as first optional positional argument.
- Hide blocks with parentheses in the name, like "(name)". This is useful for blocks that are to be required and not selected to execute alone.

## [0.2.0] - 2022-03-12

- Improve processing of file and path sepcs.

## [0.1.0] - 2022-03-06

- Initial release