destdir /home/httpd/html search /home/httpd/cgi-bin/urls.db extrafilebase /usr/local/etc/bk2site/newbase.html extrafilename new.html
In order to customize its output, you need to set up (at least) three files. indexbase.html otherbase.html and your own ~/.bk2siterc. Samples of these three files are included with the distribution and should be found under /usr/local/etc/bk2site or /usr/doc/bk2site.
There can also be extra configuration files if you wish to generate extra output (for example, you might wish to generate an xml file to be read by MyNetscape.com, or a plain text file to be used in an automate monthly newsletter). The extra template files are given with the "extrafilebase" and "extrafilename" variables in your ~/.bk2siterc
Basically, bk2site supports 5 different sections in your document. Each section refers to the type of URLS that will be output by bk2site at that point. They are:
Each section is replaced by bk2site with the appropiate urls. For example, the folder section is replaced with the URLs for the appropiate folders, while the url section is replaced by the leaf urls. Exactly how these URLs will look depends on the values you give to the different variables within each section.
The easiest way to understand how it works is to look at at the sample files. Lets look at the folders (in Yahoo! these are known as the "categories"). In the sample otherbase.html you will find something like:
<!--bk2site:begin folder--> <!--bk2site:top <table width="100%"><tr><td><ul>--> <!--bk2site:template <li><A HREF="%URL"><B>%TITLE</B></A> <I>(%LEAFS)</I> %CONDDASH <small>%COMMENT</small>--> <!--bk2site:separator </ul></td><td><ul>--> <!--bk2site:bottom </ul></td></tr></table><hr>--> <!--bk2site:cols 2--> <!--bk2site:maxnum 0--> <!--bk2site:end-->
These comments guide how bk2site generates the html for the folders. Each section must always start with the <!--bk2site:begin section-type--> and end with <!--bk2site:end-->. Within each section section-type--> there are a number of variables (e.g. top, template, section-type--> separator), each one of these variables is followed by section-type--> the value we are assigning to it.
The whole section will be replaced, in the output file, by the code generated by bk2site. The section begins with:
After the begin statemend we see a number of variables, each one is assigned a value. The function of each variable is:
There are other special comments that can appear anywhere in a template file and do not need to be within a section. They will be replaced. In their place bk2site will put some appropiate HTML code.
You can get sample copies of base files, showing how all these ugly comment can be used, at the bk2site homepage http://bk2site.sourceforge.net
There are also functions you can use. Right now, these functions are NOT available for search.pl so you should not use them in your searchurltemplate variable in your .bk2siterc.
Each function takes one or more arguments and gets replaced by the results of applying that function to the given arguments. The function are:
These functions can be nested, as in &CUT[&NOHTML[%COMMENT]|500]. The strings used as arguments to these functions cannot include the characters [, ], or | since that will confuse the parser.
You can also use HTML in the comment and it will work, but:
*********WARNING********************WARNING***********
DO NOT PUT "HREF=" IN YOUR COMMENT. There is a bug in Navigator that will make it mungle your bookmarks if you do that. The bug has been reported.
If you must, use "HREF =". The space is VERY important. I actually like to use "href =". The use of lowercase and a space makes me feel doubly safe.
*********WARNING********************WARNING***********
It also implements directory aliases. Since Navigator does not allow us to create an alias for a folder/directory, we must do it with a little trick.
If you want to alias a folder called "people" to another called "faculty", then simply create an empty folder called "people" and, in the comment field write: "<alias>faculty</alias>"
*********WARNING 2********************WARNING 2***********
Do not end any comment with ">". If you do, Netscape will swallow the last 4 or so characters of your comment. If you must end a comment with ">" use ">." instead. bk2site will get rid of that trailing ".".
*********WARNING 2********************WARNING 2***********
These comments are case-insensitive.
Remember that the folders only have a title and a creation date, so sorting them on visit or modified will not do anything to them.
You can use the -o switch, or the "outputbookmarkfile" variable in only the bookmarks that are shown on the website. That is, it excludes those that were not under the main folder and those that where under PRIVATE folders. For example
bk2site -o /tmp/published-bookmarks.html
will write out to /tmp/published-bookmarks.html. This is useful if you want to give someone else a bookmark file containing only your public bookmarks. Netscape can read these bookmark files.
The -ns switch prevents bk2site from actually generating a website. It is useful when you just want to generate the output bookmarkfile without creating a website:
bk2site -ns -o /tmp/published-bookmarks.html bookmarkfile1.html file2.html file3.html
There are a couple of problems with this output. bk2site will (1) Ignore any <HR> in the original. (2) Replace all <BR> with a space (they are there, trust me). (3) Get rid of "sort" comments, instead the bookmarks will be sorted with the appropiate order.
bk2site can merge several bookmark files, which can be in either Netscape-bookmark-file-1 format, XBEL format, or RSS format. For example:
bk2siste -o /tmp/merged-bookmarks.html ~/.netscape/bookmarks.html /home/joe/.netscape/bookmarks.html
will merge your bookmarks (~/.netscape/bookmarks.html) with Joe's (/home/joe/.netscape/bookmarks.html)
In order for this to happen, Joe's bookmarks must have some folders with the comment PUBLISH on them. Each one of these folders will be matched (using its name) with a folder on your bookmark file. The two folders, along with any subfolders, will be merged. If Joe has extra subfolders, they will be created and added.
If a new bookmark has the same (identical) URL as one already in that folder, it will be ignored.
The new bookmarks are added at the end of the folder. If you want to change the order, use the sort commands above.
This technique works fine if you and Joe are coordinating---Joe must place the PUBLISH comments in the appropiate folders. But, what if you downloaded a bookmarks.html file from the web and want to add it, as is, to your site. This can be accomplished with the <include> keyword.
You simply create a folder where you want to include the new bookmarfile, and give it the comment:
<include>/tmp/filename.html</include>
or you can use a url like
<include>http://www.multiagent.com/multiagent-bookmarks.html</include>
The file will be read and merged under that folder. Since they are merged, you can also add your own bookmarks under the folder.
To summarize: Merging can be done either by:
1- Specifying many files on the command line. The first one is the root bookmark file, the rest of them must have folders marked "PUBLISH", which will be merged with the folders on the root bookmark file with the same name.
2- Including the comment "<include>url</include>" in some of your folders, where url is the name of some other bookmark file to include under that folder. You can also add your own bookmarks under this folder.
In case you need a password on your HTTP proxy, add the following ones in addition to the lines listed above:
ATTENTION: You will have to configure your proxy before you define the channels to be displayed.
These two variables need to appear in pairs and in order, for example:
extrafilebase /usr/local/etc/bk2site/newbase.html extrafilename new.html
This means that the base file is newbase.html and the output file generated by processing this base file will be written to new.html. Note that the files you create need not be html. In fact, this feature can be used to generate any kind of text file (e.g. my.netscape channels, email newsletters, etc.).
[For the uninitated, channels are simply xml files that summarize a site's contents. Many popular news sites like slashdot and freshmeat make these available for anyone to download and use, as long as you give them proper credit. Please read and obey each individual site's use restrictions for their content.]
Another fun feature is to read and display other people's my.netscape channels in your pages. This can be done by first setting three variables in your .bk2siterc for each channel that you whish to show. The variables are.
For example, to set up a slashdot channel you would do:
channelname slashdot channelurl http://slashdot.org/slashdot.rdf channelfile /tmp/slashdot-channel
Now, this sets up the channel but it does not display it. In order to display the channel you must define a section like the ones we used before but using the channel name instead of a section name. For example, to display the slashdot channel you would place the following in one (or more) of your base files:
<!--bk2site:begin slashdot--> <!--bk2site:top <a href="http://www.slashdot.org">Slashdot News</a><ul>--> <!--bk2site:bottom </ul>--> <!--bk2site:template <li><A HREF="%URL">%TITLE</A> %CONDDASH %COMMENT--> <!--bk2site:cols 1--> <!--bk2site:maxnum 10--> <!--bk2site:end-->
Notice that since the channel files contain less information than bookmark files, the only tags you can use in your channel template are %TITLE, %URL, %COMMENT, %CONDDASH, and %HITS. Using the others will %have unpredictable results.
If you do not use the ASCII character set, or use some of those accented characters, you might find that you need to use %NOACCENTURL instead of %URL. You might also want to place the comment DIRNAME followed by a folder name, e.g. DIRNAME myasciifoldername, when the title of your folder contains characters that do not make for good folder names. The folder will instead be given the name you gave it.
If you do not use DIRNAME, bk2site will transform any non-ascii characters into their hex equivalents and use that as the folder name (not very pretty but works well as a URL).
After you compile the sources you will also find a program called "search.pl". This program is a cgi-script which reads a urls.db file automatically created by bk2site and returns the number of matches to any given query. It takes several arguments:
The program needs to be moved to your cgi-bin directory. The cgi-bin dir must have:
search.pl urls.db searchbase.html
You must also edit the program search.pl. The line (near the top):
$searchprog = "/cgi-bin/search.pl";
must be changed to point to where you put seach.pl, if different from the default value.
The search.pl program will also replace all occurences of %QUERY and %NUMBER in searchbase.html, with their appropiate values. %ESCQUERY can also be used---it is the same as %QUERY but the spaces and other special characters are escaped. See the sample searchbase.html for an example of how to use this.
If you use %IFCOMHAS(*cool*)(<img ....), with *cool* or whatever name(s) you use (e.g. *lame*, -biz-, etc). You can now do searches on this. For example, to have a "Whats cool" link, you do:
<a href="/cgi-bin/search.pl?q=*cool*&num=20">Whats Cool</a>.
To do this you must first use redirect.pl (or equivalent) to count the number of hits. To do this you must change the templates so that redirect.pl gets called with the urls you want to count, and the hits are displayed where (and how) you wand them.
A simple exaple of a template is:
<!--bk2site:template <A HREF="/cgi-bin/redirect.pl?url=%URL">%TITLE</A> %NEW <font color="red">%HITS</font> %CONDDASH %COMMENT-->
Then set
urllogfile /home/httpd/cgi-bin/urllog
to point to the urllog file generated by redirect.pl, and
#Hits older than this many SECONDS # will be ignored and purged from the #urlloggile 2592000 = 30days hitstimecutoff 2592000
Then you are ready to run bk2site.
bk2site will read the urllog file and then WRITE IT BACK but it will NOT WRITE back those hits that are before the hitstimecutoff.
# Sample .bk2siterc by Jose M. Vidal # # Multiuser support by Bradley Bell <btb@debian.org> # # http://bk2site.sourcefore.net # # This file customizes the behavior of bk2site. bk2site looks # for this file in ~/.bk2siterc. Otherwise, the file can be # given with the -f parameter. Command-line arguments override # values set in this file. # # The format is a variable name followed by a value, with some # whitespace in between. The value has to be ON THE SAME LINE # as the variable. # I show all the variables available, along with some example # values. Any line that starts with # is a comment. # # If you use either $HOME or $USER or $LOGNAME they will be # replaced with # their values as given by the shell when bk2site is run. # For example # destdir $HOME/www # sets the destination directory to be www under the user's home. # NOTE: if you run bk2site as a cron job, $USER is not defined # (on Linux) # The directory where bk2site places the files. #destdir /home/httpd/html/ # When testing you might want to set it to #destdir /tmp/ #on a multiuser system you can use: destdir $HOME/public_html/bk2site/ # This is the HTML that will be placed in place of any %NEW # tags you place in your templates. Typically, this is a gif. # Notice that you can use directives here. newgif <img src="/icons/bk2site/new.gif" alt="Added %MONTHCRE1/%DAYCRE/%YEARCRE"> # You can have bk2site reset the folders' creationtime to # be either: # maxdescendants --the max of all the urls that are descendants # of this folder. # maxchildren --the max of all the urls that are children of this # folder # normal --leave it alone. The creation time is the time you created # the folder #You set it to maxdescendants if you want to see the new.gif next # to all the folders that contain new urls. foldercreation maxdescendants # The bookmark file you want to use #bookmarkfile /tmp/bookmark.html # use the regular netscape bookmarks #bookmarkfile /home/jmvidal/.netscape/bookmarks.html # Leaving it blank tells it to use ~/.netscape/bookmarks.html bookmarkfile # The name of the file we write the published bookmarks to. #outputbookmarkfile/tmp/published-bookmarks.html # The name of the folder you want to publish #topfolder PUBLIC # leave it blank if you want all your bookmarks published. topfolder #The title you want to give to your pages title $LOGNAME's bookmarks #The folder for the news. I needs to be a subfolder of topfolder #newstopfolder News #leave empty if you do not want news. newstopfolder #Do you want bk2site to add HTML comments to its output stating # where it did the replacements? yes or no comments yes # The names of the indexbase.html and otherbase.html: # use a relative directory (not good if you are using cron) # In the RPM version, the sample files are located at /usr/local/etc/bk2site/ # indexfilename /usr/local/etc/bk2site/indexbase.html otherfilename /usr/local/etc/bk2site/otherbase.html # The names of any other filenames you want to create, along with the # name of their index files (relative to destdir). # For example, if you want an extra file with new additions you say: extrafilebase /usr/local/etc/bk2site/newbase.html extrafilename new.html #You can add more pairs, like: #extrafilebase /usr/local/etc/bk2site/tophitsbase.html #extrafilename tophits.html # Add your HTTP proxy here (won't work if moved behind the channel section!) #http_proxy <your http proxy> #http_proxy_port <the port number your http proxy listens to> #http_proxy_user <your account on the proxy> #http_proxy_password <your password on the proxy> # The channels you (might) want to display # For each channel you must define three variables # first the name you will use to refer to this channel (in the base files) #channelname slashdot # the url for retrieving it (can be either http: or file:) #channelurl http://slashdot.org/slashdot.rdf # a filename where we will write a copy of this channel. Also, if the url # cannot be read we will try to read the channel from this file. #channelfile /tmp/slashdot-channel #channelname freshmeat #channelurl http://freshmeat.net/backend/fm.rdf #channelfile /tmp/freshmeat-channel #channelname multiagent #channelurl http://www.multiagent.com/mynetscape.rdf #channelfile /tmp/multiagent-channel #channelname lwn #channelurl http://lwn.net/headlines/rss #channelfile /tmp/lwn-channel #You can add as many as you want. Note that extrafilename will pre #prepended by whatever you specify in destdir. # The number of days we leave the new.gif next to a url timecutoff 30 # The string to use for the word "Top" which appears in the navigatebar top Top # or, in another language # top Cumbre #the name of your index.html file index index.html #or in DOS world #index index.htm #The name of the file you will use for the %HITS This file must #contain a bunch of times and urls (as generated by #redirect.pl, for example). Leave empty if you don't want this #feature. bk2site will read and WRITE to this file. # urllogfile #Note that redirect.pl by default writes to a file called "urllog" # that is in the cgi-bin directory. So, typically, you might want: #urllogfile /home/httpd/cgi-bin/urllog #Hits older than this many SECONDS will be ignored and purged from the #urlloggile, 2592000 = 30 days hitstimecutoff 2592000 #If you want the tophits section to also include the news items (assuming # the news folder is under topfolder) then set this variable to "yes". #The defaults is "no", which means that the top hits only include directory # entries and not news items. tophitsincludenewsno #STUFF BELOW deals with search program #if you want to generate "urls.db", which is needed by the search program # then include here the full path. It needs to reside on the same # directory you put the "search" program. The file MUST BE NAMED # urls.db #search /home/httpd/cgi-bin/urls.db # if you do not want search, just leave it blank #search #on a multiuser system you can do: search $HOME/public_html/bk2site/urls.db #This is the path that takes us from the cgi-bin directory (i.e. the #results page) to the root (Top) of your bookmarks. If your bookmarks #reside on / and your search in /cgi-bin/bk2site, then: #searchtorootpath ../../ If your Top is in a subdir then #searchtorootpath ../subir/ If the search is done on a different #machine, then #searchtorootpath http://machinewithbookmarks.org/bookmarks/ #on a multiuser system you can do: searchtorootpath /~$LOGNAME/bk2site/ #The url template to use for the search results. This should probably # be the same template you use for the urls in the otherbase.html file. # BUT: note that search.pl automatically prepends every url with an <li> #searchurltemplate <A HREF="%URL">%TITLE</A> %NEW <font color="red">%HITS</font> <A HREF="http://www.hotbot.com/?clickSrc=search&MT=%URL&SM=url&DC=50"><img src="/icons/bk2site/closeup.gif" border=0 alt="Who points to it?"></A> %IFCOMHAS(*cool*)(<img src="/images/cool.gif">) %CONDDASH %COMMENT searchurltemplate <A HREF="%URL">%TITLE</A> %NEW %CONDDASH %COMMENT
Contributors include:
August Hoerandl - wrote check-meta.pl. added "date" tag.
Bradley Bell - patch to support multiuser setups.
Daniel Barrero - patch for handling flat file systems
Reinier Mostert - patch to handle bookmark aliases.
Oliver Obst - provided the autoconf scripts.
Hannes Faestermann - patch for new.html
David Cancel - cool ideas and code for improving search.pl
The program homepage is at http://bk2site.sourceforge.net