documentation overhaul

9 files changed, 359 insertions, 0 deletions

diff --git a/docs/CHECKLIST b/docs/CHECKLIST
new file mode 100644
index 0000000..6c4a4f7
--- /dev/null
+++ b/docs/CHECKLIST
@@ -0,0 +1,43 @@
+THIS FILE IS DEPRECATED.
+TODO: make sure that all this gets merged into the documentation
+
+This file contains all things that are/should be working.
+When you have been working on something, use this list to verify that you did not cause any regressions (things that worked before, but got broken).
+Also, when you finish a new feature, add it to this list so that everyone knows about it, can play with it, and can test it.
+
+Also testers and interested people can use this list to see what uzbl is about, and report problems if they find items listed here that do not work.
+
+* commandline switches --uri "http://.." and --config behave as expected
+* --help and -? should show help
+* config loading: loading of settings, bindings and all behavior settings (including handlers) from config file
+* fifo: type this in your shell after starting uzbl:
+  echo 'uri http://www...' > <fifo path here>'
+  where <fifo path> is printed on stdout during program startup.  Also "back" , "forward", "refresh" (basically all actions defined in `static Command commands` should behave as expected)
+  On one of Dieter's pc's, using the fifo only works once.  Please report issues with the fifo if you find any.
+* socket: uzblctrl -s <socketname> -c <command> (same commands as with fifo)
+* history logging: the script as defined in sample config should write history entries (date,time, url) into the file defined in the script (by default ./extra/history.sh and /tmp/uzbl.history)
+* invocations of external commands: you'll see it on stdout everytime uzbl calls an external script/program. the invocations should work and none of the arguments passed should be null/garbage/.. .
+  all should be valid strings and contain things like the pid, fifo file, config file,.. (see README for details).
+* XDG_CONFIG_HOME and XDG_CONFIG_DIRS (+ default values) fully supported and working (looks for a config file called 'uzbl/config').
+* --uri can be specified without http:// -- if missing will be prepended.
+* Downloading of files with handler script
+* Open in new window. all (valid) args passed to your uzbl instance are passed also to the new instance, with the obvious exception of --uri.
+* internal keybinds to change things work (as loaded by config)
+* vimlike command and insert modes behave as expected
+* always_insert_mode and modkey work too
+* status bar can be toggled on and off, if off, some info (mode) will be put in title bar. position of statusbar (top/bottom) also works
+* backspace erases last edit character. esc erases entire command buffer (if in comand mode)
+* commands with parameters (keywords): any key sequence that ends with underscore _ expects parameter. Undersore marks exact location where parameter begins. Command part can have %s that will be replaced with parameter.
+  This only works for internal commands. not when spawning external scripts.
+        :o _ = uri %s - user needs to type :o www.google.com
+        :O_ = uri %s - user needs to type :Owww.google.com
+* when typing command, press insert (paste X cliboard) or shift insert (paste primary selection buffer)
+* proxy, user agent and other network settings can be set in config
+* scrolling works, with configurable keys (vi-like in sample config)
+* searching:
+  /_ = search %s <-- hilight all
+  ; = search <-- jump over all hits
+* run javascript on curent page through "script" command.
+* variable replacement in user agent.
+* basic keyboard link hilighting (numbering) and following. will be improved more
+* support overriding background color of statusbar and pango markup format to customize the statusbar in terms of which variables, separators, colors, fonts, etc
diff --git a/docs/CONTRIBUTING b/docs/CONTRIBUTING
new file mode 100644
index 0000000..8aba17e
--- /dev/null
+++ b/docs/CONTRIBUTING
@@ -0,0 +1,24 @@
+Users
+
+Right now, the best way to contribute to Uzbl is to use it, hang around in
+our IRC channel, and tell us when things break. If you're feeling more
+adventerous, you can use one of the development branches and give bug
+reports and suggestions straight to the developer in charge of that, so the
+same problems don't occur when they get merged into the master branch. Have
+a look at the CHECKLIST file to see all the stuff that is supposed to work.
+Play around with the configs and scripts and see if you can improve things.
+
+Developers
+
+If you don't feel like just sending bug reports, by all means dive into the
+code and clone the code to start hacking. (github makes this really easy
+with their "fork" concept).  But it's usually a good thing to tell us first
+what you want to do, to avoid unneeded or duplicate work.
+
+
+
+VALGRIND PROFILING
+add this to Makefile header: CFLAGS=-g
+recompile
+valgrind --tool=callgrind ./uzbl ....
+kcachegrind callgrind.out.foo
diff --git a/docs/FAQ b/docs/FAQ
new file mode 100644
index 0000000..e3652f9
--- /dev/null
+++ b/docs/FAQ
@@ -0,0 +1,72 @@
+FAQ
+---
+
+### I just installed uzbl but it doesn't do much.  I can load one uri and quit the program but that's it.
+You did not load a configuration.  Uzbl does not create a default config file on startup like some other programs do.
+Because we want to give you the freedom to place your config where you want, and to use a config or not.
+Have a look in /usr/share/uzbl/examples/configs.
+Use the --config parameter or save your config as $XDG\_CONFIG\_HOME/uzbl/config to have it auto-loaded.
+
+### Where is the location bar? How do I change the URL ?
+Uzbl has no location bar.  All changes to the uri (editing of current uri, typing new uri, loading of uri from bookmarks/history/...) happens *outside* of uzbl.
+Have a look at the sample scripts in /usr/share/uzbl/examples.  Most of our examples use dmenu which is a nifty little tool to pick an item from a list of items (very
+useful with history/bookmarks) with a limited set of keystrokes.  see man dmenu.
+You can also use it to make edits to a uri (press tab to load it into the
+search field) or type a url from scratch, though dmenu
+is not the most suitable editor. We're looking into a better way to make edits.
+Zenity is also an option, if you want copy-paste support.
+
+### Where are the widgets (forward, back,.. button etc)
+Uzbl's layout only contains what you really need to see.  we only have a statusbar, which even can also be disabled.  There are no buttons, but we do
+have lots of keybinding possibilities.
+
+### What?  No support for bookmarks/history/downloads/cookies/... ? Your project sucks!
+We do not support *management* of those things, because we believe a browser should only do browsing.  We are firm believers in the unix philosophy.
+You have to look at the bigger picture.  In fact, we do support all these things.  Take bookmarks as an example:
+
+ * we support keybinding and spawning external programs, so you can bind a key to spawn any script you want
+ * Your script receives properties such as the current url, window title etc.
+ * You can then call a tool such as zenity to prompt for any more information you may want to specify (tags,...)
+ * You have the freedom to store the bookmarks in whichever format you want. (plaintext, sqlite, any database, on a remote system, in version control, ...)
+ * To load a bookmark, you trigger another script which invokes a tool such as dmenu to let you pick a bookmark.  Your script can send the command to load the url to uzbl very easily by using the socket or fifo interface.
+ * To manage your bookmarks, you can use whatever you want, depending on how you store them (simple text editor, database interface, ... )
+
+These ideas are something we want to consistently apply throughout the entire application.  (Even more, throughout our entire desktop environment)
+In fact, we actually ship various sample scripts and some sample configs that make it easy for you to implement your workflow.
+
+### Why can't I type anything in forms?  How does the keybinding work?
+You are in command mode, not in insert mode.
+
+* command mode: you can trigger actions inside uzbl with minimum amount of keypresses (eg 'b' to go back, 'ZZ' to quit etc)  (see config examples), but not to type actual text into forms, because all your keypresses are interpreted.
+* insert mode: after going into insert mode (by default this is the 'i' binding from inside command mode), your keypresses are not interpreted but passed on, so you can enter text into forms.  Press Esc to go out of insert mode.
+
+The above method is called "modal" as inspired on VI.  If you don't like
+this you can easily change this:
+
+* enable always\_insert\_mode in your config.  You will always be in insert mode.
+* configure a modkey.  Since your keypresses are not interpreted anymore to trigger actions, you need a modkey to do things (eg alt+'b' to go back instead of just 'b' from command mode)
+
+This method is how many applications work.
+
+Both have their pro's and cons.  We don't want to force anyone in using
+either, so by tuning the modkey and always\_insert\_mode settings you can pick
+whichever method you like, or both at the same time (command mode, insert mode, and the modkey to perform actions while in insert mode)
+
+### Why do you depend on gtk?
+Uzbl itself doesn't use much gtk stuff (only the statusbar) so we could do without gtk.  But Webkit needs a widget toolkit to create widgets (think javascript popups, html forms etc).
+Officially, it also supports QT and wxwigdets.  There are also some unofficial patchsets floating on the interwebs for the EFL and FLTK toolkits.  One could argue we don't need no popups or fancy form widgets and you could have a point, but
+we prefer being reasonably assured that things work as they are supposed to rather then using some obscure patchset which may be incomplete, broken and/or badly designed, or wasting time ourselves in what is not our core objective.
+Note that we do *not* depend on any Gnome libraries such as gconf.  _That_ would be something worth complaining about :)
+
+### Do you support flash? javascript? Ajax?  Recent html/css/.. standards?
+Yes, Webkit takes care of all of that.  Not that we like all of these, but you can use them if you want.
+
+### Does the world really need another browser?
+We did try a lot of browsers, and we do not suffer [NIH](http://en.wikipedia.org/wiki/Not_Invented_Here).
+We believe that the approach taken by way too many browsers is wrong.  We do not want browsers that try to do everything,
+instead we prefer a system where different applications work together, which gives plenty of advantages.
+We also like open source.  We take a lot of things from other projects and we also try to contribute to other projects.
+
+
+### What? You call all of this user-friendly?
+Yes.  If you don't agree, don't use it :)
diff --git a/docs/INSTALL b/docs/INSTALL
new file mode 100644
index 0000000..270c343
--- /dev/null
+++ b/docs/INSTALL
@@ -0,0 +1,63 @@
+Arch Linux
+----------
+[Arch Linux](http://www.archlinux.org) is our distro of choice, and the distro we use for testing.
+
+You can find a [PKGBUILD](http://aur.archlinux.org/packages.php?ID=25972) on the AUR, which installs the latest
+from the master branch. You can edit the PKGBUILD to change to any other
+branch you want.
+
+From source
+-----------
+	$ git clone git://github.com/Dieterbe/uzbl.git
+	[ $ git checkout master/experimental ] # optional. see below
+	$ cd uzbl
+	$ make
+	$ sudo make install
+If you want to remove uzbl again, you can issue:
+
+	$ make uninstall
+
+Dependencies
+------------
+* git (for downloading)
+* pkgconfig (for Make/gcc)
+* libwebkit 1.1.4 or higher
+* libsoup 2.24 or higher (dep for webkit/gtk+)
+* gtk 2 something something
+
+Optional/Recommended
+--------------------
+The following tools are quite useful, and some of them are used in the
+sample scripts:
+
+* dmenu (with vertical patch
+* zenity
+* bash 
+
+File locations
+--------------
+After installing - using either method - you will find:
+
+* /usr/bin : uzbl [and uzblctrl]
+* /usr/share/uzbl/docs/ : documentation files included with uzbl. (readme, checklist, .. )
+* /usr/share/uzbl/examples: sample scripts, config files and a sample data (boomarks, .. ) 
+
+You will probably want to change the scripts to behave more like you want, so copy the scripts to your home dir. If you save your config as
+$XDG\_CONFIG\_HOME/uzbl/config (this expands to ~/.config/uzbl/config on most systems) it will be recognized automatically. You can also pass the path to
+the config file with the --config parameter.
+
+You're free to store your personal stuff where you want, but we think the [XDG basedir spec](http://standards.freedesktop.org/basedir-spec/basedir-spec-0.6.html)
+is very useful for keeping a clean home directory, so we recommend:
+
+* $XDG\_CONFIG\_HOME/uzbl/config* (~/.config/uzbl/config on most systems): config file
+* $XDG\_DATA\_HOME/uzbl (~/.local/share/uzbl on most systems): bookmarks file, history file. and "scripts" directory with scripts
+
+Git Repo's & branches
+--------------------
+* Main official repo:
+  http://github.com/Dieterbe/uzbl
+- master -> uzbl stable branch
+- experimental -> bleeding edge stuff that may break. after QA Dieter merges into his master
+
+* Most contributors & developers also have their clones on github (http://github.com/dusanx, http://github.com/Barrucadu/uzbl, ...).
+  They may be developing specific features, which get merged into Dieters experimental branch
diff --git a/docs/TODO b/docs/TODO
new file mode 100644
index 0000000..39ea849
--- /dev/null
+++ b/docs/TODO
@@ -0,0 +1,67 @@
+* implement all the ideas from README
+* implement a more advanced dmenu alike that behaves like FF's awesomebar and where you can search in url + window title, with support for Xorg copy paste
+  ideal uri editor: awesome mode like FF, some keyb shortcuts (erase search string, go to end/begin of string,..), history (if you patch dmenu to be in vertical mode and you order correctly, that's it), support copy paste
+  isolate the search field feature from midori and make into a separate dmenu-like-but-more-powerful program?
+* recognize -h with GOption?
+* implement a vimperator-like link following scheme.  but let user pick his favorite characters to construct the "link identifiers" with.
+* add a keybind to hand the current url to an external scrips, so you can edit it and/or store it in the primary and secondary clipboards
+* clean up our structures for settings + sane defaults + don't iterate over struct to fill hasthtable. rather fill hashtable directly
+* implement getting feedback from socket
+* select/fork based instead of the pthread stuff -> drops dependency, more lightweight.
+* scrolling: make page up and page down configurable.
+* show % of location in statusbar/title if page doesn't fit entirely on view.
+* make default size configurable, and optional
+* on uzbl.org commits overview: add date+time and repository
+* how to handle different content types? (text-plain, image/png, application/pdf,... maybe a map of content-type to uzbl/command
+  xdg already has a spec for this i think
+  different "opening" modes (open as configured vs ask before opening)
+  integration with download and new window thingies?
+* blinking cursor when not in insert mode is confusing.  i suggest dimming it's color if possible
+* open in new window -> uzbl: Fatal IO error 11 (Resource temporarily unavailable) on X server :0.0.
+* note about merging better then patches. because commit names, and code can have changed in meanwhile. also github makes the process quite easy
+* check that in new version pageup etc works also in command
+* tab key to jump between input fields should probably work in both insert and command mode
+* allow to name a uzbl instance, name fifo/socket after it. use xorgwindow id as fallback
+* allow to tag , to group instances together
+* allow users to customize order, separating, colors,.. of items in statusbar using pango markup thing
+* mention bugtracker in readme, move tickets from github to new bugtracker
+* default value for fifo_dir, socket_dir so we don't need to if(fifo_dir). it will always be set to something?
+* change User-agent to contain uzbl + build date or something. + why is there "Mozilla" in the string
+* backspace key to pop characters from (multichar) command
+* optional logging of http requests&responses with ip/hostname and port. -> how to implement? handler? stdout? (through a socket so you know what corresponds to what?)
+* bench/optimize fifo vs socket performance. measure delays.  minimize forks. does glib use a shell? how does it detect the shebang line?
+* cookie support.  storing seems to work, but not yet sending
+* "remember account settings" support. but how? configure post data per site? regex match eg '^bbs.archlinux.org' ?
+* http_proxy env var not recognized. libproxy (used by libsoup) should handle this http://mail.gnome.org/archives/libsoup-list/2009-February/msg00018.html
+* support ssl. do ssl certificate & exception management similar to how we do cookies
+* improve DCOMMIT macro.  what if WC is dirty? what if user downloaded tarball without .git?
+* DARCH is not correct (should be at runtime)
+* when loading page foo.com, it can have img src=http://bar/..., uri in uzbl will be set to foo. we must pass bar to cookie handler
+* set default statusbar. line 1284
+* keybinds to open "next" or "previous" by looking for next/prev links and/or looking for numbers in the uri we can inc/decrement
+* variable replacing for title bar in "statusbar_visible" and statusbar_invisible states
+* get rid of config files -> send everything as commands to stdin on program launch -> you can change config at runtime.  
+  how to handle launching new windows? serialize state with tpl? (can also be useful for session saving), fork()ing? if so, how do we handle Xorg window id's?
+* handler for (broken) ssl certs.
+* bring readme up to date and put it on website
+* handlers for mailto: and maybe other thingies?
+* only_on_put
+* configure script
+* a variable that holds the page state: loading, pending, seen. this can be shown in titlebar/statusbar and used for multiple instances management
+* proxy_url is not a good var name. it's not a url.
+* regex style page searching? so you can do 'or' and 'and' things. flags like case sensitive etc.
+* can we export uri/state/tag/.. as xorg window "properties" or "attributes" or do we need to put them in titlebar and parse that?
+
+* config: check in the default place (XDG_CONFIG_HOME/..) for a config file, and if needed, do file reading (interpret line by line). not ini-based.
+* readd the --config flag to allow entering the same commands either through the file/stdin/fifo/socket
+* check for real command name, not just the first letter.
+* let users attach handlers to the most common events/signals in uzbl.
+* write little script to open new urls with the urxvt url thing +document.
+* add Rob/anydots trees on website
+
+SOMEDAY:
+check if we can make the settings loading less hard coded. eg( keep a list of all settings, and for each one, try to load it)
+figure out caching with webkit and in general how we can speed up everything
+figure out how webkit intercepts key input
+make "disable insert mode" (esc key) configurable
+keywords don't work for external commands. is this a problem?
diff --git a/docs/multiple-instances-management b/docs/multiple-instances-management
new file mode 100644
index 0000000..da55002
--- /dev/null
+++ b/docs/multiple-instances-management
@@ -0,0 +1,41 @@
+"multiple instance management" or: how to manage multiple open pages/instances of uzbl.
+
+The way we do MIM in uzbl will be better then what you can do with tabs in other programs.
+Tabs are just one specific gui implementation which aids in some aspects of "multiple instances management" but not all.
+We can get the same and even better features functionality wise, without limiting ourselves to the particular implementation that tabs are.
+We use a "use-case"/tasks driven approach, unlike tabs which is a "lets do this, and then see how we can use it" approach.
+
+The approach we are implementing in uzbl is like this:
+- 1 page per instance.
+- ability to spawn new windows ("open in new window"), keybinds for having the new window focused or not.
+- each instance keeps track of it's page state (loading, loaded but not seen, loaded and seen)
+- ability to "tag" instances (inherited when forking new instances) (eg tag "work" or tag "personal") so you can keep related instances together.
+  The tag can be made visible in the title / statusbar.
+- allow user to make keybinds to focus the next/previous:
+  - window
+  - unseen/seen window
+  - unseen/seen window from the same tag
+  - unseen/seen window from tag <foo> (dmenu)
+- have a program that lists all instances (optionally filtered, or categorized by tag, state, ...) and allow user to select one from it using smart matching techniques which require a minimal amount of keystrokes. (dmenu with vertical patch is great for this)
+
+Here's an overview what do tabs really do, whether we really need it, and if can improve it?
+
+
+* tabs keep multiple open pages together when moving the window other tag/workspace
+   -> in an ideal workspace, you assign windows to a tag/workspace and keep them there.
+   -> in practice, when I was still using Firefox, I noticed I moved my "group of bundled pages" (eg a FF window) sometimes, to aid in copy pasting between browser and another app, or doing something while watching/reading a webpage.
+      This means however I only really need that specific page, not the others. -> With the suggested approach, you can easily temporarily move one window.
+* tabs keep an oversight of which pages you have open
+   -> you should "know" more or less know which pages you have open.  The gui is clutter.  It should however be possible to show the list "on demand".
+   -> a horizontal list of tabs is also not very readable and lacks useful categorisation/grouping.
+   With the suggested approach we will have each url below each other, which is far more readable, and we can use additional text or color themes to denote the state/tag/...
+* tabs aid opening new pages in the background which you will open later and provide shortcuts to go to the previous/next tab.
+   -> our approach is more powerful.
+   -> With tab based browsers you can usually configure if you want new tabs to open "immediately after the current tab", or "at the end of the list". This is much less flexible/usable then what we are building.
+
+Conclusion
+---------
+I think our method is better then tabs.  Bonus advantages:
+* single uzbl instances are simple to implement, no added clutter.
+* one crashing instance does not affect the rest
+* all this stuff is implemented outside of uzbl.  You can even decide to not use it and you won't even know the feature was ever there.  (though i guess most people will want this).
diff --git a/docs/performance b/docs/performance
new file mode 100644
index 0000000..0f9763f
--- /dev/null
+++ b/docs/performance
@@ -0,0 +1,9 @@
+The usage of external scripts causes some slowdowns. (forking new processes, running interpreted scripts,...)
+It can be interesting to keep an eye on this, though I don't expect major problems.
+
+** History file size/performance
+each new pageload -> fopen(history_file, "a"), fwrite one line, fclose.
+I use utf8, so unless you use characters that are "special" (chinese etc) each character takes 1 byte.
+So, assume each entry is about 80 chars, you visit 100 pages per day (?), and you wonder when your history file will be 50MB big:
+(50 * 1000 * 1000 ) / ( 80 * 100 ) = 6250 days or 17 years.
+There is code to run a benchmark in the 'extra' dir.  For results & interpretation, see http://dieter.plaetinck.be/poor_mans_dmenu_benchmark
diff --git a/docs/url-editing b/docs/url-editing
new file mode 100644
index 0000000..fae96c7
--- /dev/null
+++ b/docs/url-editing
@@ -0,0 +1,7 @@
+* All url changing/editing happens outside of uzbl
+
+Use cases:
+* load url from history/bookmarks (dmenu-vertical is great for this)
+* edit current url or one from history/bookmarks: dmenu-vertical is relatively good for this, though it would be better if we had:
+- minimal editing helper key shortcuts (eg to go to beginning/end of input buffer)
+- copy/paste support
diff --git a/docs/widgets b/docs/widgets
new file mode 100644
index 0000000..dc7127a
--- /dev/null
+++ b/docs/widgets
@@ -0,0 +1,33 @@
+
+* statusbar? (the bar you see in pretty much every gtk program at the
+* bottom. eg firefox)
+  consumes too much space (if always visible) for the little it is used.  (+
+* you can put only 1 message in it at a time!)
+  -> option 1: no statusbar at all. when hovering over a link (or pressing
+  -> key to preview the url without changing page) -> show url in tooltip on
+  -> page.
+  -> option 2: toggle visibility of statusbar on/off when hovering over a
+  -> link. since it's at the bottom I don't think it will disturb too much.
+* viewing progress/state of pageload?  most programs use statusbar for this.
+  -> option 1: titlebar can show a percentage when it's loading a new page.
+  -> option 2: toggle a statusbar everytime we start loading a new page.
+* uri bar -> yes, even though we can write stuff to the fifo, it can still
+* be convenient to change the url manually and stuff, so a widget in uzbl
+* itself is good.
+* tabs -> yes. you don't have to use them, but you can.
+* back/forward/.. buttons? -> no: use keyboard shortcuts.
+* searching in a page? not sure.. maybe we can abuse the statusbar for that
+* too.
+  eg toggle it on when the user wants to search for something and then do
+* searching in some vim-like fashion.
+  we don't need a gtk text entry widget, just a feedback display of what the
+* current command is.
+* scrollbar? no: use keyboard shortcuts.  we should however have some info
+* about the page length and where we are.
+  -> option 1: put a percentage in the window title
+  -> option 2: everytime you hit a key to change position, temporarily make
+  -> a statusbar visible and put the percentage in the statusbar.
+  what will we do with pages who are too wide? horizontal scrolling?
+all of the above goes in 1 bar at the top of the program. there should be a
+key to toggle visibility of it and one to toggle visibilety + focus on the
+entrybar at once.