Main

Lisp Archives

August 5, 2002

Common Lisp Controller

I write Lisp programs with an attempt to have them run correctly on as many Lisp implemenations as possible. To do this requires that I test my programs on multiple Lisp platforms. (The main platforms that I use are AllegroCL Enterprise, Lispworks Professional, and CMUCL.)

Peter Van Enyde's Common Lisp Controller package is quite useful for this sort of testing. The basic idea is that you install your source code and system files in a standardized location. Then in your Lisp environment, just give to command (require <pkg>) and the files in your package will be compiled if necessary and loaded into your environment. Common Lisp Controller gives each implementation it's own hierarchical directory structure to store it's compiled code.

Currently, Common Lisp Controller is best supported with Debian GNU/Linux. A nice integration is that when ever you a CLC-aware Lisp package with Debian's package tools, Common Lisp Controller will automatically compile and install the files in the package. Further, when the package is removed, CLC will remove the source and compiled code from the system. I've been working with Peter recently on CLC and it is becoming quite powerful on Debian development (sid/unstable) systems.

September 19, 2002

ASDF

ASDF, Another System Definition Facility, is a tool to automate the compilation and loading of complex Common Lisp systems. I've recently converted all of my system definition files from the venerable mk-defsystem 3.3 to ASDF.

I've been quite happy with the results. Especially now that I've added ASDF support to Common Lisp Controller (CLC). Rather than having to struggle with supporting logical pathnames on multiple Lisp implementations, the ASDF files with CLC don't have to embed any pathname at all.

This also eliminates the difference between a CLC'd package and a non-CLC'd package. Previously, a CLC'd package had to put the location of the CLC root directory in the system definition file. Thus, the one needed a different system definition file for a CLC and non-CLC package location. Now, with ASDF files, that is not necessary. The same .asd file can work from both inside and outside of CLC. This is good.

LML

Lisp Markup Language, LML, is a new package that I've released under the GPL license. I use it for creating all of my web sites. I find it is much more productive to use LML that to write straight HTML. If you try LML, consider sending me a message with your thoughts on the package.

Wow, lemonodor posted an announcement about this package before I did. Kudos to him.

September 20, 2002

CLisp and CLC

Great, another Common Lisp implementation now works with Common Lisp Controller. I've written support for clisp to use CLC. This works quite well with ASDF-based CLC systems. As I wrote before, I like using CLC to test my code with multiple Common Lisp implementations. Now, clisp can be added to that list of testing implementations.

September 25, 2002

ILC 2002

I was asked to post an announcement for the International Lisp Conference 2002. It's looks to be a great conference. You can read about it at their web site.

I wish that I could attend!

September 26, 2002

CLisp's Suprising Results

I've moved all my web sites to Lisp source code and use LML to create the XHTML files for the web site. Using Common Lisp Controller, it's easy for me to test LML on a large number of CL implementations. As usual, testing LML on multiple platforms revealed that my code relied on some non-ANSI CL functionality of particular implementations. In the process of testing on multiple implementations, I found those instances of non-ANSI code and now have nice, portable code as a result.

I then ran a benchmark to compare the speed of implementations for this particular application. What surprised me is how well CLisp performed. CLisp, as opposed to other implementations, compiles to platform-independent byte-code rather than the native code of the CPU. In the benchmark results that I posted, Clisp performs brillantly. And, since CLisp works on nearly all CPU platforms that have a C-compiler, it's a good choice for me since I run Debian GNU/Linux on multiple CPU platforms.

To be fair to the other CL implemenations, my web site generation benchmark uses mostly uncompiled source (LML) code as the input. I believe that SBCL, in particular, spends a lot of time working on compiling the code before executing it.

September 27, 2002

The CLC Hit Count

The short answer is 6.

I've recently adopted the openmcl package for Debian and have added Common Lisp Controller support for that compiler. At this point in Debian, CLC is supported by:

This is great: for any of the above implementations in Debian, I can just give the command (REQUIRE :LML) and have the LML package automatically compiled if necessary and then loaded.

September 30, 2002

UFFI & CLSQL for OpenMCL

John Desoi wrote UFFI support for MCL and OpenMCL. Today, I updated that code and merged it into the main UFFI source. Also, I've reworked CLSQL a fair bit so that it now runs on OpenMCL.

CLSQL and UFFI now support AllegroCL, Lispworks, CMUCL, and OpenMCL. At some point, I'd like to add support for SBCL.

Portable Aserve in Debian

I've uploaded Portable Aserve into Debian's development distribution. It integrates with Common Lisp Controller, so one can just type (require :aserve) and have acl-compat and aserve automatically compiled if necessary and then loaded.

October 15, 2002

Benchmarking CL with UMLisp

I've done some benchmarks for various CL implementations for my UMLisp package. I use UMLisp mostly in the context of a web-based server. Since the Unified Medical Language System database is 8GB of SQL data, queries can take seconds to minutes. So CL implementations with true multi-threading (Allegro and Lispworks) are suitable for the UMLisp web server while CMUCL and SBCL are not.

The first benchmark is concerned with loading SQL objects via CLSQL and creating CLOS objects while the second benchmarks printing using metaclass-driven formatting.

The free CL implementations which not not have true multithreading , CMUCL and SBCL, do great on this benchmark. Allegro trails significantly in the narrow-benchmark of formatting via metaclass-stored formatting data.

ImplementationLoadPrint
CMUCL4.6s45s
SBCL5.1s61s
Lispworks11.7s46s
Allegro13.2126s

October 17, 2002

Scieneer Common Lisp

Scieneer CL interests me. I've downloaded an evaluation version and have ported UFFI and CLSQL to it.

I'm especially intrigued with it's multitreading capabilities. Since SCL is based on CMUCL, it should do well in the UMLisp benchmark test that I recently ran. Given that SCL is truely multithreaded, unlike CMUCL or SBCL, it would be appropriate for use as the UMLisp web server. While CMUCL and SBCL are fine for running web sites that have quick page generation, there are some UMLisp queries could take hours and return gigabytes of data.

October 27, 2002

CL-Reversi

I read Peter Norvig's excellent Paradigms of Artificial Intelligence Programming book when I first started learning Lisp last year. This book includes an excellent chapter on implementing the game of Othello®. For learning purposes, I reworked to the code to add significant optimizations and a CLIM-based graphical user interface.

I noticed this week that Peter has posted a liberal license for his code from his books. So, I packaged up my version and have published it at http://reversi.b9.com/.

October 30, 2002

UMLisp

I've recently released my Lisp interface to the Unified Medical Language System (UMLS). The UMLS is a meta-thesaurus for medical terminology. The UMLS is large: the raw database is 4 gigabytes of ASCII data containing nearly 900,000 concepts and over 2 million text strings.

The UMLisp web site has details of some of the interesting techniques employed. There is an online demonstration system that can be used to browse the UMLS which uses XML and CSS for results.

I've spent significant time developing this product. Intentionally, I do not include any documentation. I am hoping to sell documentation and support -- it'll be interesting to see what happens with that.

November 6, 2002

Hyperobject

During the development of UMLisp, I developed a small object representation scheme designed to support Unified Medical Language System (UMLS) objects. The UMLS is densely relational. A single concept can have thousands of subobjects and related objects. Using an object representation library allowed a simple, unified way to represent relationships between the 30 distinct object types in UMLS.

For my UMLS browser, I needed to print objects in ASCII text, HTML, and XML formats with optional printing of field labels and hyperlinks to related objects. I've packaged this object representation library and have made it freely available. This library is available under a BSD license and can be downloaded from ftp://ftp.b9.com/hyperobject/".

November 25, 2002

New Hyperobject

I released a new version of hyperobject that allows slot options to be specified with the slot definition rather than the class definition. An example demonstrates the difference in the schema:

Old Version
(defclass person ()
  ((last-name :type :string)
   (first-name :type :string)
   (dob :type integer)
   (addresses))
  (:metaclass hyperobject-class-old)
  (:ref-fields (last-name find-person-by-last-name))
  (:subobjects addresses)
  (:fields last-name first-name (dob format-date))

New Version

(defclass person ()
  ((last-name :type :string :reference find-person-by-last-name)
   (first-name :type :string)
   (dob :type integer :print-formatter format-date)
   (addresses :subobject t))
  (:metaclass hyperobject-class)
  (:print-slots last-name first-name dob))

This is much improved and will simply adding new object representation features.

April 12, 2003

Multi-threading with SBCL

Besides working with SBCL on my sb-aclrepl module, I've also been spending more time with the latest versions of SBCL since taking over the maintainence of the Debian SBCL package.

The latest versions SBCL now support multi-threading. This is available in the CVS tree of SBCL in the 0.pre8 versions. Though this feature is still in a pre-release state, it is available for widespread testing. This feaure is quite nice for using SBCL to serve web pages. Though SBCL has been used successfully serving fairly static web pages, for web pages that involve significant computation or database access, having multiple processes run simultaneously is quite important.

I've uploaded a new Debian package called sbcl-mt which contains a multithreaded version of SBCL. It is currently in the unstable distribution of Debian and will move to the testing distribution in the next few weeks, users can have their choice of running either the single or the multi-threaded version of SBCL.

Since the .fasl format is not compatible between the single and multi-threaded versions of SBCL, the Debian Common Lisp Controller package maintains separate directories for sbcl and sbcl-mt compiled files. Also, to help people who want to have both sbcl and sbcl-mt running on their Debian system, I've changed the default fasl extension for sbcl-mt to be .faslmt.

SBCL's Surprise

Since working on a AllegroCL-compatible REPL (read-eval-print-loop) for SBCL, I've been spending more time with SBCL. SBCL has focused on their source-code portability and understandability rather than performance optimizations. That's all nice, though benchmarks in SBCL have often underperformed other CL implementations.

I've finished a new set of benchmarks with UMLisp and the latest version of hyperobject. This version of hyperobject adds processing of slot options and also uses more features from the Art of the Metaobject Protocol. Hyperobject has an improved print architecture that is much faster and as well as more flexible and extensible. I've revised my benchmark program so results are not directly comparable to previous published benchmarks. I've also tested Scieneer Common Lisp (SCL) which hyperobject fully supports.

The win for SBCL in this benchmark is huge. It'll be interesting to investigate why this is.

ImplLoad (user/total)Print (user/total)
AllegroCL 6.28.3/13.85.5/5.7
CMUCL 18d+6.3/123.6/5
Lispworks 4.27.0/123.5/4
SBCL 0.pre8.581.8/6.41.8/1.9
SCL 1.1.14.7/10.24.5/4.8

April 22, 2003

Units package

I've submitted a new Debian package for the Common Lisp Units software. It's does a very fine job with simple and complex units conversions.

Hyperobject work-around

With Christophe Rhode's expert help, we found the spot where Hyperobject was failing with the latest CVS versions of CMUCL. I've been able to work around this CMUCL bug by temporarily removing a feature in Hyperobject that UMLisp is not currently using.

This work-around is great because I've been wanting to try the UMLisp benchmark with the MOP enhancements that Gerd Moellmann has added in the post-18e CMUCL release. The preliminary benchmarks show a significant improvement in the new version of CMUCL. I'll post a full set of benchmarks after the CMUCL bug gets fixed.

Thanks, Christophe!

64-bit addressing

The day has finally come for the 64-bit Opteron CPU; reviews are posted in the usual sites. I'm very excited about chip as I've been bumping against the 1.6GB process limit with 32-bit architectures for several years.

I've already run Debian with a 64-bit toolchain with x86-64 software emulation using Bochs. Though I'm excited about the 64-bit hardware, it really won't do much good for me until there are x86-64 Common Lisp implementations. I'm hoping that such a port will be created later this year.

April 29, 2003

MOP Improvements

I'm quite indebted to Gerl Mollermann and Christophe Rhodes for their outstanding work on improving the MOP (Metaobject Protocol) implementations in SBCL and CMUCL. The latest set of hyperobject improvements exposed bugs in both SBCL and CMUCL. Gerd and Christophe find and squash bugs in a matter of hours. With their efforts, the MOP implementations in these Common Lisp implementations have reached the high level of AMOP compliance that I've found with AllegroCL. Along with better AMOP compliance and big bug reductions, Gerd's 18e+ MOP incorporates big performance optimizations in CMUCL.

With their improvments, I've generalized the attributes slot metaclass in my kmrcl helper package to work with CMUCL, SBCL, and SCL in addition to the long-standing AllegroCL support.

Latest MOP Benchmark

With Gerd's and Chrisophe's MOP fixes, I've been able to run the UMLisp benchmark against the latest versions of CMUCL and SBCL. I've double the number of runs in this series compared to the last published benchmarks. The big change is CMUCL with Gerd's new CLOS/MOP optimizations. CMUCL 18e+ has doubled it's speed in the print benchmark compared to 18d+. Nice work!

I've included SBCL-MT, the multi-threaded version of SBCL, in this series. It nicely performs nearly identically to it's single-threaded version.

ImplLoad (user/total)Print (user/total)
AllegroCL 6.217.0/27.711.0/11.1
CMUCL 18e+2.8/11.93.0/3.2
Lispworks 4.212.4/239.1/10
SBCL 0.pre8.1183.1/12.53.9/4.2
SBCL-MT 0.pre8.1183.0/12.63.7/4.5
SCL 1.1.18.8/20.19.2/9.7

May 9, 2003

Refactoring

William Newman of SBCL fame recently recommended Martin Fowler's book Refactoring. After reading it, I immediately found successful uses for his techniques. I took the opportunity to look back at some of my early Lisp code to see how refactoring could improve that code.

I found a few recurring themes that I was able to improve:

  • Long Functions
    These are functions that do more than one thing breaking Chris Riesbeck's Cardinal rule of Functions: One Function to A Function. One of the main problem with long functions is that are they are hard to read in review because they do too many things. Another problem is that by having all of the code in one function, the subcomponents of that function can't be used by other functions. A clue that such function do too many things, besides their length, is that they may contain comments or white space separating the various tasks. This problem is fixed by splitting the function into multiple functions, each with descriptive names. Then, comments splitting the sections are no longer needed, but rather, are embeded in the function names themselves.
  • Duplicated Code
    Efficiently removing duplication of code is one area where Common Lisp excels. As an example, in UMLisp there is a file which reads data from the a large number SQL tables and creates various CLOS objects from that data. My first pass at the code resulted in a large number of functions with lots of duplicated code. Their structure was in the form of:
    WITH MUTEX SQL CONNECTION
      CREATE A SQL QUERY via FORMAT statement
      SUBMIT QUERY
      LOOP EACH ROW
         DESTRUCTURING-BIND FIELDS
         COLLECT MAKE-INSTANCE of CLOS OBJECT
    
    To refactor this code, I created a macro to handle to generation of the SQL query and iteration on each row of the results. The new code looked like:
    COLLECT-FROM-SQL-QUERY [query parameters]
      MAKE-INSTANCE of CLOS OBJECT
    

    In this design, the all of the field names required for the query were passed to the macro which created the query string and also setup the destructuring-bind. This refactoring resulted in my file shrinking 50%. Up to this point, other languages besides Common Lisp could have approximated this refactoring. But, the below shows Common Lisp brilliantly shining.

    After the first pass, benchmark tests showed doubling in runtime. I tracked this performance decrease to the overhead of creating the SQL query string. In the original code, the SQL query string was hard-coded whereas in the new version it is generated dynamically from the input parameters. By modifying my new macro, I was able to move nearly all of the query string generation to compile-time rather than run-time. That is impossible with most other languages. With this change, my new code was only 10% slower than my original code.

    In the process of micro-optimizing this centralized macro, I found a place where I could change a (format nil "~D" number) to (write-to-string number). With that one change, my new code was now 10% faster than my original code. This was complete success! Looking back, I could haven take that one optimization and applied it to my existing code, but that would have required duplicating the modification in 40 functions. With elimination of the code duplication, that optimization was only coded once.

    I believe this is a recurring pattern: by refactoring to centralize a commonly used block of code, the centralized function may have increased overhead to support the many ways that it is used. However, optimizations can then be employed to improve the efficiency of that centralized function and improve the overall performance of the software.

  • Not taking advantage of combined iteration/element-testing functions
    When I first starting programming in Common Lisp, I was not aware of the very useful functions that iterate over a sequence while applying an predicate and testing the result. Employing such functions as some, nonany, and every allowed me to greatly simplify some code. For example,
    (defun any-element-invalid-p (list)
      (dolist (elem list)
        (when (invalid-p elem)
          (return-from any-element-invalid-p t)))
      nil)
    
    simplifies to
    (defun any-element-invalid-p (list)
      (some #'invalid-p list))
    
  • Consing inefficiencies
    In my early Lisp code, I didn't pay as much attention to consing overhead when writing code. As a result, I inadvently created a few bottlenecks. A potent example is when I naively wrote a function like:
    (defun list-to-comma-delimited-string (list)
      (let ((output (format nil "~A" (car list))))
        (dolist (elem (cdr list))
          (setq output (concatenate 'string output "," (format nil "~A" elem)))))
    
    That highly-consing code was found to be a bottleneck in a system when it was called with a list of 15,000 elements. A simple rewrite suggested by Larry Hunter uses a feature of the powerful FORMAT function and ran 100 times faster:
    (defun list-to-comma-delimited-string (list)
      (format nil "~{~A~^,~}" list))
    
    So while atrocities like the above example can be a bottleneck, I've learned when the overhead of consing may result in better code. In a rewrite of another function, I found a function that avoided consing and looked like:
    (let ((lst (partially-processed-elems)))
      (dotimes (i (length lst))
         (declare (fixnum i))
         (let ((elem (nth i lst)))
           ...lots of elem testing/processing...
           (when (not-fully-processed elem)
              (setf (nth i lst) (fully-process elem)))
      lst)
    
    However, this function could never be a bottleneck as it was only called once at the initialization of a software system. I was able to simplify the code with eliminating the list index and shrink the function at the cost of creating a new list:
    (loop for elem (partially-processed-elems)
       collect (...lots of elem processing...
                    (if (not-fully-processed elem) 
                       (fully-process elem)
                        elem)))
    

A bottleneck

Using Allegro's profiler, I found a bottleneck in one tiny function that I had written a long time ago. Despite its simpleton nature, this function is important because it is invoked once for each line of output in the UMLisp print benchmark. By changing the poorly-implemented function
(defun indent-to-level (n &optional (stream *standard-output*))
  (format stream (format nil "~~~DT" (+ n n))))
to the more sane
(defun indent-to-level (n &optional (stream *standard-output*))
  (declare (fixnum n) (optimize (speed 3) (safety 0) (space 0)))
  (dotimes (i (the fixnum (+ n n)))
    (declare (fixnum i))
    (write-char #\space stream)))
I got a marked speed increase in the print benchmark

ImplOld Time
(user/total)
New Time
(user/total)
AllegroCL 6.211.0/11.18.2/8.3
CMUCL 18e+3.0/3.22.2/2.3
Lispworks 4.29.1/105.8/6
SBCL 0.8alpha.0.223.9/4.22.8/3.0
SBCL-MT 0.8alpha.0.143.7/4.52.7/3.2
SCL 1.1.19.2/9.75.1/5.5

This is a 26-44% overall speed increase by changing this one little function tucked away in an old library file. The point proven once again is that there is no adequate substitute for profiling: a major bottleneck may exist in some tiny, forgotten function.

May 14, 2003

Avoiding Allegro's FORMAT

I recently used AllegroCL's profiler to learn why AllegroCL has done so poorly on the print benchmark. I found that AllegroCL is unusually slow when using FORMAT to print strings and numbers.

When I first designed hyperobject I assumed that it was faster to use a single FORMAT call with multiple values being printed at once rather than issuing sequential write commands. However, this is not true if writing the individual components can be made much faster than FORMAT's writing of individual components.

After some preliminary benchmarks with AllegroCL showing a 30x speed increase in writing short string using WRITE-STRING rather than FORMAT "~A", I reworked hyperobject's object display. Now, rather than caching a large format string and creating a function that returns the data values of all of the fields of an object, the program now compiles a series of commands to print the individual components (along with the usual formatting, labeling, and hyperlink elements).

Even though my typical object display has moved from a single (long) format command to dozens of write invocations, I've nearly tripled the speed of the printing in AllegroCL. Speed increases for other implementations range from 5-41%.

ImplOld Time
(user/total)
New Time
(user/total)
Increase
AllegroCL 6.28.2/8.32.9/3.0183%
CMUCL 18e+2.2/2.32.1/2.25%
Lispworks 4.25.8/64.1/441%
SBCL 0.8alpha.0.262.8/3.02.5/2.512%
SBCL-MT 0.8alpha.0.142.7/3.22.5/2.78%
SCL 1.1.15.1/5.54.6/4.811%

SBCL-AMD64

I posted a message on comp.lang.lisp today asking others to contribute to a project to produce an AMD64 port of SBCL. I've already committed money to fund a portion of the project. I'm hoping that other people who require a freely available, high-quality, 64-bit Lisp implementation will contribute a tax-deductable donation as well.

July 10, 2003

cl-modlisp

The UMLisp web site recently switched from using Franz's AllegroServe to using Marc Battyani's speedy mod_lisp Apache module. This module provides an interface from Apache to a Lisp server process. One of the very nice features of mod_lisp is that it supports persistant connections using HTTP/1.1's Keep-Alive protocol.

The downside of mod_lisp is that it is a very low-level interface and requires the creation of a listener process running in Lisp. cl-modlisp is a high-level interface that I wrote and made freely available that makes using mod_lisp a pleasure. It has number of useful features for process control and web page generation.

July 11, 2003

Current UMLisp Benchmarks

I've written about my benchmarks and optimization efforts for my UMLisp library. Last month I spent a fair bit of time with Allegro's profiler. I optimized routines which used significant amounts of time. In some cases, this involved rewriting built-in Lisp functions with my own optimized replacements. So, my kmrcl library now has functions like prefixed-fixnum-string. No exactly what I wanted to do, but that optimized routine was many times faster than a frequently used invocation of (format nil "C~7,'0D" id).

Below are the initial and current print benchmarks. While I've previously lamented AllegroCL's speed, AllegroCL is now the fastest on the print benchmark. Its top rating was helped, no doubt, by using its profiler to guide my optimizations.

ImplInitial TimeCurrent TimeIncrease
AllegroCL 6.28.20.79930%
CMUCL 18e+2.21.2576%
Lispworks 4.2.75.81.97190%
SBCL 0.8.1.302.81.6471%
SCL 1.1.15.11.7200%

July 13, 2003

Webactions

I've been working on an object-oriented, high-level web framework based on cl-modlisp, hyperobject, and LML2. It's an interesting time for Franz to announce their Webactions package.

Though my approach is quite different than Franz's, it was interesting to consider how they handled some of the common web framework issues.

July 18, 2003

Testing Portable URI

I've recently uploaded Puri (Portable universal resource identifier) and tester (a regression suite). These are based on open-source packages from Franz. Like most Franz open-source software, they initially used many AllegroCL specific features. After my porting efforts, Puri successfully executes all 126 regression tests.

Puri and tester are available for download from my web server.

December 24, 2003

Symbolics OpenGenera

I recently purchased a copy of Symbolics OpenGenera and built an Alpha 21164 Tru-64 system for running the environment. I'm still learning the environment, but my initial impressions are quite positive.

Lisp IRC Logs

Recently, I wrote an IRC logging bot as well as a log browser. I've published the logging bot as the irc-logger library. The browser is not yet feature complete: I'm still working on search capabilities and more highlighting options. Even so, the browser is quite usable even now. I'm using the very nice cl-irc library for the logging bot. I also wrote an conversion program using the cl-ppcre library to import lisp logs extending back to Sep 2000.

December 27, 2003

OpenGenera Benchmarks

I ran the Gabriel benchmarks included with OpenGenera (SYS:EXAMPLES;GABRIEL-BENCHMARKS.LISP). I'm running on a 533MHz Alpha 21164 with 512MB RAM. I've published the results. Ranier Joswig sent me his MacIvory3 results for comparison.

Comparing the results, OpenGenera on the Alpha system is 3 times faster on most benchmarks except for floating point. For floating point code, OpenGenera is up to 38 times faster (FFT benchmark) than MacIvory3. Given the Alpha's FPU, this is not at all surprising.

December 28, 2003

Searchable IRC Logs

I've finished adding indexing and searching to my IRC log browser.

December 30, 2003

More IRC Browsing

I've added graphing of message counts, colored message by author, cookie-based auto-login, and selectable start page.

January 13, 2004

RLC Circuits

I just finished and released the initial version of rlc, a Common Lisp RLC circuit solver.

I've been toying around with Laplace transformations and testing the results with gnucap. To assist verifying the analytic solution with the gnucap simulation, I wrote a Common Lisp program to plot the output of the various solution formulas.

While I considered porting my C++ plotter library from CTSim to CLIM, I found that invoking xgraph from Lisp worked so well for this application that I couldn't justify the effort of the port.

Continue reading "RLC Circuits" »

February 3, 2004

New Meme Features

Several people have requested having a link to the current day's log for a channel. I've added that feature along with a filter to display only the URL's that are contained in the messages of a day's log.

As an example, the URL to view today's Lisp logs is http://meme.b9.com/now?channel=lisp. The URL to view the URLs contained in today's Scheme messages is http://meme.b9.com/now?channel=scheme&url-only=t.

February 10, 2004

Going Universal

To assist with localizing the Meme browser to a user's local time, I've converted the base logs to Universal Time Coordinate files. Currently, I have channel view and searches correctly displaying in user's local time. Users of Meme's clog and f00f virtual servers, however, only display in UTC at the moment.

April 7, 2004

New CLSQL version

With the orphaning of UncommonSQL, Marcus Pearce did a fine port of the CommonSQL API compatibility layer to CLSQL. I've integrated his port into CLSQL and added compatibility with AllegroCL's metaobject protocol. The original CLSQL without the new features is now exported by the CLSQL-CLASSIC system which continues to be supported.

April 16, 2004

ODBC Support

Yesterday I finished adding ODBC support to CLSQL. That raises the number of CLSQL database backends to 6.

CMUCL large files to the rescue

I'm working on importing a new version of the UMLS for UMLisp. UMLS continues to grow and now one of my intermediate files is larger than 2GB.

Allegro and SBCL, my two primary platforms, don't support large files on my Linux AMD64 box. But, Raymond Toy gave me the tip that CMUCL does support Large (greater than 2GB) files. He's right - CMUCL came to my rescue and allowed me to import the new UMLS version.

May 25, 2004

CLSQL Developments

Marcus Pearce and I have been spending a lot of time on CLSQL development.

I released version 2.11.1 today with full Oracle support (via OCI) as well as AllegroCL AMD64 support. CLSQL now supports 7 native database backends.

June 9, 2004

Bill's CLSQL Experience

Bill Clementson has an knack for effectively presenting software packages. I've gotten a good introduction to several software packages reading his web log. Recently, Bill discussed his experience with CLSQL (part one, part two).

August 3, 2004

CLSQL 3.0

I've released CLSQL 3.0. This release of CLSQL is a major rewrite compared to version 2.0. New features include:

  • Full CommonSQL backward compatibility with native documentation while retaining and adding many improvements beyond the CommonSQL standard.
  • Extensive (233 tests) regression suite
  • Addition of ODBC and Oracle backends

I wish to acknowledge Marcus Pearce's significant contribution to this release. He has performed the initial port of Uncommonsql to CLSQL as well as the initial regression suite and most of the new documentation. He has contributed to many of the redesign decisions and new code for CLSQL.

Continue reading "CLSQL 3.0" »

December 4, 2004

SBCL-AMD64 Disassembly

With Dan Barlow's SBCL AMD64 initial backend and Juho Snellman's fixes, I took some time to work on its disassembler. It's been interesting to read the AMD64 technical documentation as well as see the wide registers being used by SBCL.

CL-USER(1): (disassemble 'car)
; 0109DB80:       .ENTRY "top level local call CAR"(LIST)     ; (FUNCTION # *)
;       B0:       488F45F0         POP QWORD PTR [RBP-16]
;       B4:       488D65C0         LEA RSP, [RBP-64]
;       B8:       488BD9           MOV RBX, RCX
;       BB:       488BF2           MOV RSI, RDX
;       BE:       4883FB08         CMP RBX, 8
;       C2:       752C             JNE L0
;       C4:       488BC6           MOV RAX, RSI
;       C7:       40240F           AND AL, 15
;       CA:       403C07           CMP AL, 7
;       CD:       7526             JNE L1
;       CF:       488BC6           MOV RAX, RSI
;       D2:       488B50F9         MOV RDX, [RAX-7]
;       D6:       488B4DF0         MOV RCX, [RBP-16]
;       DA:       488B45F8         MOV RAX, [RBP-8]
;       DE:       4883C103         ADD RCX, 3
;       E2:       488BE5           MOV RSP, RBP
;       E5:       488BE8           MOV RBP, RAX
;       E8:       FFE1             JMP ECX                                                     

February 28, 2005

Yay, Peter Van Eynde

Peter has kindly offered to take over maintainence of most of my Debian lisp packages. At this time, he's agreed to take:

cl-aima
cl-ansi-tests
cl-asdf
cl-binary-types
cl-blowfish
cl-csv
cl-defsystem3
cl-environment
cl-f2cl
cl-faq
cl-gd
cl-geodesics
cl-grt
cl-html-template
cl-infix
cl-inflate
cl-integrate
cl-interpol
cl-irc
cl-iterate
cl-jpeg
cl-md5
cl-menusystem
cl-meta
cl-metering
cl-net-telent-date
cl-paip
cl-pdf
cl-plus
cl-png
cl-port
cl-portable-aserve
cl-postoffice
cl-ppcre
cl-ptester
cl-readline
cl-rsm-bitcomp
cl-rsm-bool-comp
cl-rsm-cache
cl-rsm-delayed
cl-rsm-filter
cl-rsm-finance
cl-rsm-fuzzy
cl-rsm-genetic-alg
cl-rsm-gen-prog
cl-rsm-memo
cl-rsm-mod
cl-rsm-modal
cl-rsm-mpoly
cl-rsm-queue
cl-rsm-rand
cl-rsm-random
cl-rsm-rsa
cl-rsm-string
cl-rss
cl-rt
cl-screamer
cl-scribble
cl-split-sequence
cl-ssl
cl-statistics
cl-syslog
cltl
cl-ubf
cl-unit
cl-units
cl-who
cl-xmls
sbcl
xml-to-sexp

Thank you very much!

April 19, 2005

Finally, another Common Lisp library

It's seems like a long time since I started a new Common Lisp project. I've been getting tired of creating ad-hoc spreadsheets to calculate some moderate complex photography formulas and started to use the Lisp REPL for evaluating some of these formulas.

That worked so well that I started formalizing these formulas into a new Common Lisp library: cl-photo. At the moment, the library has rather extensive field of view, angle of view, depth of field, hyperfocal distances, and circle of confusion calculations with flexible length units.

I'll be adding more formulas, web references, and documentation. After that, I may add an interactive web calculator, though, the calculations would likely be best done in Javascript rather than CGI or a web application. CGI has the disadvantage that starting a Lisp session has high overhead and a web application has the disadvantage of yet another daemon to maintain 24x7. A Common Lisp to Javascript translator would be a handy thing...

April 23, 2005

Online Photography Calculator

I've greatly improved cl-photo since it's initial release earlier this week. I wrote a CGI calculator to compute some common (and not so common) photography values.

February 2, 2006

Diverging From Upstream

I have written and currently maintain PURI, a ported version of Franz's open-source URI (Uniform Resource Identifier) library. The Franz library uses a number of Allegro specific optimizations. I've tried to remain true to their optimizations by using similar optimizations on other Lisp implementations where possible.

However, the is one non-Allegro specific optimization that has caused a number of people some trouble: that the library expects input strings to be simple strings rather than generalized lisp strings. That has caused at least 3 people to mention the issue to me. I pondered the best solution for a bit. Likely, the optimum result would be to write a macro that emits generic functions specialized to both simple and generalized strings. However, I took a simpler route: I removed the simple-string specific optimiztions (such as using schar rather than char). I expect the reduction of trouble for library users outweighs the run-time overhead.

A few bug fixes including accomodating a change in the function of SBCL's shink-vector, and PURI 1.4 is now available.

August 30, 2006

New CLSQL Platforms

I've released version CLSQL 3.7.0 with support for more platforms. All tests pass on these platforms now.

  • SBCL Win32
  • OpenMCL AMD64
  • CLISP Win32, Cygwin, Linux i386, Linux AMD64

Special thanks for the CLISP support to CMUCL for its LOOP's packge to allow the CLSQL loop extensions and to CFFI-UFFI-COMPAT for its CLISP support.

September 2, 2006

New Hyperobject Version

I've released version 2.10.0 of hyperobject. This version improves AMOP compatibility and the test suite. The release also adds CLISP support . Since Hyperobject requires CLSQL and now that CLSQL supports CLISP, I was able to add support for CLISP in this release.

New UFFI Version

I've released UFFI version 1.5.16. It adds support for the shared library file type on Cygwin.

September 4, 2006

CLSQL Version 3.7.1 Released

Provides a minor bug fix for AllegroCL's lowercase lisp reader, mlisp.

Also, special thanks to Christophe Rhodes, frequent contributor to SBCL's MOP, for his excellent suggestion in response to a question for improving CLSQL's MOP internals: CLSQL object definitions use custom slot types. For example, a CLSQL slot may have :type (varchar 10) specified which gets translated to a lisp type of (or null string). Rather than parsing and then re-storing the type atrribute of a slot in compute-effective-slot-definiton, Christophe suggested performing the type parsing in initialize-instance :around of the CLSQL direct-slot-definition object. Then, the real type attribute is stored in the both the direct and effective slot definition from the beginning.This is more AMOP complaint since AMOP doesn't specify that one may change the type attribute of a slot. This is clearly seen since CLSQL no longer needs to modify OpenMCL's ccl:type-predicate slot attribute after the type was changed in compute-effective-slot-definition.

March 9, 2007

Cluck - New Common Lisp Library

I've been working on several new microcontroller designs and found that I was using a calculator too often to compute optimal parameters for the microcontroller timers. So, I built a small library to automate the computations and now it's quite quick to pick a optimal master clock speed, timer prescalar, timer compare value, and UART divisors.

The library is called cluck (Common Lisp μcontroller ClocK calculator) and is available at http://files.b9.com/cluck. Its initial functions include:

  • displaying range of frequencies and periods for 8, 16, and 32-bit timers values for a given clock speed
  • displaying prescaler, compare values, and error percentage for a desired millisecond interrupt period
  • displaying UART divisors and error percentage for common serial port speeds

About Lisp

This page contains an archive of all entries posted to Kevin Rosenberg in the Lisp category. They are listed from oldest to newest.

AI is the previous category.

Medical Imaging is the next category.

Many more can be found on the main index page or by looking through the archives.

Creative Commons License
This weblog is licensed under a Creative Commons License.