Caution: my English is far from perfect. (Русский тоже не всегда хорош).

blog

Friday, 22 August 2014

Semantic Versioning is Not the Solution

People often think they can introduce incompatible changes in their library API, and just increase major version number, as semantic versioning proposes, to save the library clients from problems.

It is not true.

Consider a dependency tree:
my-application
  web-server 1.1.1
    commons-logging 1.1.1
  db-client 1.1.1
    commons-logging 1.1.1
  authentication 1.1.1
    commons-logging 1.1.1
Now commons-logging changes its API incompatibly and is released as commons-logging 2.0.1. Authentication adopts commons-logging 2.0.1 while other libraries still depend on 1.1.1:
my-application
  web-server 1.1.1
    commons-logging 1.1.1
  db-client 1.1.1
    commons-logging 1.1.1
  authentication 1.1.2
    commons-logging 2.0.1
Now my-application is broken, because the dependency tree includes two versions of commons-logging which share packages, class/functions names, and thus can not be loaded simultaneously.

When you release an incompatible API this way, you essentially split the world of dependent libraries into two parts: the ones depending on the old version, and ones depending in new version. Libraries from the first part can not be used together with libraries from the second part.

A better way to introduce incompatible API is to release it as a new library, for example commons-logging2, or new-logging. Make it possible to use the new library simultaneously with the old one, e.g. it should have new package name.
Doing so will protect clients in majority of cases.

If we are releasing new library for new API, there is no need for such a thing as "major version number".

NB: in some module managers, most notably in javascript, there are no global package/class names on which different versions of a library can interfere. But in majority of programming languages that problem exists.

Thursday, 20 March 2014

МИД Беларуси считает, что Крым был, есть и будет.

Saturday, 25 January 2014

Какие похожие картинки. Очень надеюсь, что до похожего результата дело не дойдет.







Monday, 20 January 2014

How many more years it will take Chrome and FireFox to learn display text properly?

These are original size screenshots of some page heading.
I think such bad results may be achieved if Chrome and FireFox rasterize first and scale after that, instead of scaling first and then rasterizing.
To reproduce, here is the code: <h1 style="font-size: 300%; color: #4f81bd; font-style: italic; font-family: 'Times New Roman', Times, serif;">WINTER ACTION</h1>

Update: Many people reported FireFox and Chrome render fonts OK for them. HN comment and tnm91 suggested that DirectWrite may be disabled, due to outdated video driver. It was true, I updated diver and DirectWrite become enabled, but rendered hadn't changed. Then sulliwan suggested to check CliearType - I enabled it and now FireFox and Chrome render text OK.

Thanks for the help!

Monday, 23 September 2013

issue 9999

Have just opened another GAE issue, and it got number #9999
It's pretty. The next reporter will get number 10000.

Friday, 3 May 2013

Указ 60 и ЕЭП

Давно думаю, не противоречит ли указ №60 принципам Единого экономического пространства Беларуси, России и Казахстана?

Вроде у нас свобода торговли, и вдруг раз - на территории Беларуси при оказании услуг с помощью информационных систем могут применяться только информационные сети, системы и ресурсы "национального сегмента сети Интернет, размещенных на территории Республики Беларусь и зарегистрированных в установленном порядке". Значит ли это, что например Российским системам, провайдерам хостинга и т.п. вход на Белорусский рынок закрыт?

Sunday, 14 April 2013

Github support for org-mode in READMEs is unsatisfying

Github support for org-mode in READMEs is unsatisfying - no code coloring and sometimes it even fails to render to html at all.

I am migrating all my README files to markdown. Googled also this tool for automatic conversion: https://github.com/alexhenning/ORGMODE-Markdown. Will try it next time.

Apart from READMEs I also would like to represent TODO.org files online. While org -formatted READMEs are more or less readable in gitbub's HTML representations, TODO.org files are not readable at all.

Saturday, 9 February 2013

Quest for Immutable Static Files Hosting

For cl-test-grid I need an online storage for static files - library test logs.

The requirements:

  1. Permissions:
    • Upload new file - anyone (because the tests may be run by anyone)
    • Delete or modify existing file - forbidden for public.
  2. Current upload rate is around 50 000 small files per month, usually uploaded during several days after new Quicklisp is out. In the future it may increase to 100 000 or more. Size of each file is <= 100 KB.

After file is uploaded it must be available by an HTTP URL.

None of the online file storage services I saw satisfy my permissions requirements. Usually if user can upload file, he can delete/modify it. Therefore every of the solutions below include an intermediate server application which enforces my access rules.

I have considered several variants:

  1. Google App Engine Blobstore

    The first solution I found and it served me for more than a year. The free usage quota includes 5 GB of blobstore, which I hoped will be enough for years.

    But it shown some disadvantages. I faced some bugs: #7619, #8032. Also every blob upload takes several Datastore operations, thus exhausting the free Datastore quota. Also the blob keys are 164 characters long, making the log URLs horrible: http://cl-test-grid.appspot.com/blob?key=AMIfv97wo4FQxGLBZOagyZyLZCqwMWavAfwsByKxjq8QiJQ5rIzEggGwGJ_kH2qRZLMb8N_el8aKIpLDbnr67Pxcy9r8RFKmBnjTQ1B44yaCcyZWtO2CSbBliyAINvoI41_R8uA8hoPia-yXPdlmADiJcavCCgpHGA

    To make the URLs shorter I save blobkey in a Datastore Entity, and use the integer entity ID as a reference to the log:

        Entity shortKeyEntity = new Entity("ShortKey");
        shortKeyEntity.setProperty("blobKey", blobKey);
        datastore.put(shortKeyEntity); 
        return shortKeyEntity.getKey().getId();
    That way the URLs look better: http://cl-test-grid.appspot.com/blob?key=789714. But this increases amount of Datastore operations. Also, the Datastore creates some internal indexes for the entities, and which take huge amount of space. I in total have 1GB of log files, but it takes 4GB of Datastore to keep the mapping from blob keys to short IDs:

    So my application exceeds the free quota by several parameters and the paid plan must be used. It is > $108 per yer. Expensive for just publishing 1GB of static files.

  2. Google App Engine + Google Cloud Storage

    Cloud Storage is an Amazon S3 clone. I migrated to it from App Engine Blobstore recently.

    As you choose file names yourself there is no problem with long keys. Also Cloud Storage doesn't involve any App Engine Datastore operations.

    There is an API to access it directly from App Engine, although it is experimental. And I faced a bug #8592; solved it by a work-around.

    Other difficulties to solve were:

    • Handling of multipart/form-data based file uploads, because Google App Engine does not support servlet API 3.0 yet. And Apache Commons FileUpload library doesn't work out of box in App Engine, because it tries access file system.
    • Java Servlets on App Engine should handle every request in 30 seconds time-out. The servlet is killed if exceeds the time-out.
    • Writing files one-by-one to Cloud Storage via the Java API is relatively slow.

    Having solved all this by customizing Apache Commons FileUpload to keep the uploaded files in memory while parsing the request body and writing to Cloud Storage from multiple threads I now have a solution allowing to upload 300 logs in one request. And with the free 5GB provided by Google for the fist Cloud Storage project, the payments for this service will most likely be zero.

  3. Amazon S3 + Heroku.

    This is the next thing I'll do if I have any problems with CloudStorage. A small app at Heroku creating pre-signed S3 URLs and returning them to client. The client uploads files and after the pre-signed URLs expire he can not modify the files.

    Amazon only allows to upload one file per PUT request. With the price of $0.01 for 1000 requests it may cost me around $10 a year.

    A tempting property of this solution is that the Heroku app may be written in Common Lisp thus getting rid of Java servlets in cl-test-grid.

Tuesday, 5 February 2013

Multicore Forth processors provide Go-like concurrency primitives in hardware

In the Go Concurrency Patterns presentation Rob Pike demonstrates how unbuffered channels are
enough for many concurrency tasks. (NB: use Left/Right arrow keys to scroll the presentation)

It reminded me of the Forth chips produced by Chuck Moore and colleagues.

The current version contains 144 cores per square centimeter chip. The machine language is Forth and
each core is equipped with its own little data and control Forth stacks, making it a fully fledged
independent computer (that's why the more precise term is "multi-computer chips" rather than "multi-core").

The cores talk to each other via communication ports. Writing to a port suspends the core until
the peer reads the value. And vice-versa.

This semantics corresponds to the Go channels.

The chips have other interesting properties. Quoting the doc:

A computer can read from multiple ports [Corresponds to Go's select]
and can execute instructions directly from those ports.

FINE GRAINED ENERGY CONTROL: ... The read or write instruction is automatically 
suspended in mid-operation if the address [one or more of communication ports and I/O pin] is inactive,
consuming energy only due to transistor leakage currents, resuming when the address becomes active. 

NO CLOCKS: Most computing devices have one or more clocks that synchronize all 
operations. When a conventional computer is powered up and waiting to respond 
quickly to stimuli, clock generation and distribution are consuming energy at a huge rate 
by our standards, yet accomplishing nothing. This is why “starting” and “stopping” the 
clock is a big deal and takes much time and energy for other architectures. Our 
architecture explicitly omits a clock, saving energy and time among other benefits.

Read more at the company website.

As Rob Pike says, the channel-like concurrency primitives are not new. It is interesting to see
them implemented in hardware.

Monday, 3 September 2012

Linux memory over-commit.

Linux memory over-commit. Previously I've only read about it, but now I am bitten by it. When cl-test-grid agent (a CCL program) starts ECL and ECL starts gcc, linux on my VPS kills someone of them (usually CCL). Sometimes it just doesn't allow to fork new process. Why do CCL and other Lisps allocate so large heap at startup?

Tuesday, 17 January 2012

maybeComma

I once made a small invention: the simplest way to join values in 
a collection using comma as a separator:

    result = '';
    maybeComma = '';
    for (val : collection) {
      result += maybeComma + val;
      maybeComma = ',';
    }

If collection = [1, 2, 3], then result = '1,2,3'.

I invented this almost 10 year ago, and it was useful many times. There are 
environments, like shell scripts, where there is no Arrays.toString method. 
And even C++, at the times when I programmed on this language, didn't have any
such "join" function in the standard library (that's why I needed to invent it; 
BTW, it's interesting, does C++ have join today... quick googling makes 
impression it doesn't...).

I never saw more simple implementation. All the implementations I saw have 
an IF inside the loop:
  
    result = '';
    for (i = 0; i < collection.length; i++) {
      if (i > 0) {
         result += ',';
      }
      result += collection[i];
    }

Here we also needed to change the "for each" construct to a "for by variable i", 
to determine if we are at the first iteration or not.

The maybeComma solution does not depend on the iteration construct at all.
Therefore it is easier to adopt. For example joining two collections.
In the maybeComma approach we just copy/paste the loop without rethinking
the logic, it always works the same:

    result = '';
    maybeComma = '';
    for (val : collectionA) {
      result += maybeComma + val;
      maybeComma = ',';
    }
    for (val : collectionB) {
      result += maybeComma + val;
      maybeComma = ',';
    }

In the IF approach we again need to change the control constructs by 
introducing a boolean flag:

    result = '';
    oneDone = false;
    for (i = 0; i < collectionA.length; i++) {
      if (oneDone) {
         result += ',';
      }
      result += collectionA[i];
      oneDone = true;
    }
    for (i = 0; i < collectionB.length; i++) {
      if (oneDone) {
         result += ',';
      }
      result += collectionB[i];
      oneDone = true;
    }

As far as I remember this understanding of how to avoid logic by switching to 
different data values came to me from ThinkingForth - the idea of how to use
the maybeComma variable to avoid the IF in the solution was a result of conscious 
attempt to apply what is taught in this book. 

Tuesday, 27 September 2011

Private Members in Javascript

It is usually advised to use closures to emulate private class members in javascript. Like this:

function MyClass(param) {

   var privateVar = 1;

   function privateMethod(a) {
      return a + 2;
   }

   this.publicVar = param;

   this.publicMethod = function(x) {
     return x + this.publicVar - privateVar;
   }
}

var my = new MyClass();
my.publicMethod(1);

I find this approach inconvenient for various reasons. Right away I can remember at least 4:

  • Private members are not visible in debugger (FireBug)
  • You can't access private members programmatically (during development I often need to experiment and access private members)
  • Public methods which need access to private data should be defined inside of the closure, we can't add them to the class prototype.
  • In the code private and public members are referred differently: without this for private and with this for public. If I want to make some previously private method public, I need to find all the references and change method() to this.method(). And often the correct this is not available, because we are in some deeper function/closure, where this refers different object (for example we are in a callback of AJAX call). So, making a method public can not always be done mechanically.

We may try to workaround the last issue by defining even public methods as closure-local functions and then publishing them explicitly:

function MyClass(param) {

   // ...

   function publicMethod(x) {
     return x + publicVar - privateVar;
   }

   function publicMethod2(x, y) {
     return publicMethod(x) + y;
   }

  this.publicMethod = publicMethod;
  this.publicMethod2 = publicMethod2;
}

But that way descendant classes can't override the public methods. In short, the closure approach is unsatisfying.

I have an idea which looks for me better than using closures to emulate private members.

The class implementation may have everything accessible via this (i.e. public):

function MyClass(param) {
   this.privateVar = param;
}

MyClass.prototype.privateMethod = function (a) {
      return a + 2;
}

MyClass.prototype.publicMethod = function(x) {
     return x - this.privateVar;
}

var my = new MyClass();
my.privateMethod(2);
my.publicMethod(1);
Then, if I want to restrict class clients from using the private implementation details (and feel that code comments are not enough for that) I can provide them only public interface:
var my = {
   impl: new MyClass(1),
   publicMethod: function (x) {return this.imp.publicMethod(x);}
}

// This is how clients normally use it:
my.publicMethod(1);

// But it is also possible to check the private implementation for debugging and development purposes:
my.impl.privateMethod(2);

This separation of encapsulation from classes resembles Common Lisp.

And the difference between obj.method() and obj.impl.method() is similar to Common Lisp: package-name:func-name for exported symbols and package-name::func-name for not-exported symbols. Client can access private implementation if he wants, but the syntax difference ensures he knows what he is doing.

Depending on the situation, we may create such interface objects manually, or create a little function which wraps any object into an interface:


// Whole the code below is tested, you can run it e.g. in FireBug console.

function as(impl, interface) {
  var wrapper = {impl: impl}; 
  for (prop in interface) {
    wrapper[prop] = makeImplCaller(prop);
  }
  return wrapper;
}

function makeImplCaller(functionName) {
    return function() {
        return this.impl[functionName].apply(this.impl, arguments);
    }
}

// This is the public interface
var Interface = {
  funcA: null, // (a)
  funcB: null, // (a, b)
}

// The implementation of the interface
function Impl (x, y) {
    this.privX = x;
    this.privY = y;
}

Impl.prototype.funcA = function (a) {
    return this.privX + a;
}

Impl.prototype.funcB = function (a, b) {
    return this.privX - a + this.privY - b;
}

// How to provide the inteface to client
var impl = new Impl(5, 10);
var i = as(impl, Interface);

// And client uses it:
i.funcA(1);
i.funcB(7,5);
// inspecting private details
i.impl.privX;
i.impl.privY;

Beware, in some cases the fact that implementation is wrapped into a wrapper might affect the program. If we compare o1 === o2, it might happen that o1 is impl, and 02 is a wrapper, and we get false when true is expected. This is the kind of problem many Java programmers face sometimes (in Java ecosystem various decorators, AOP, proxies, etc. are often used).

I had no time to think further about this approach. For example, in the above code making i instanceof Interface returning true.

But in general I like the idea.

Thursday, 2 June 2011

Unlucky day for OpenID

Tried today some small changes in cl-openid and faced various problems (in 3rd parties! verified it by unchanged cl-openid and also by testing with other online services).

Blogger OpenID provider is just broken (http://www.google.com/support/forum/p/blogger/thread?tid=120fff3e2b1a061d&hl=en)

Livejournal OpenID provider started to violate the spec (as I read it) by returning error response with HTTP status code 200 OK. (See 5.1.2.2. Error Responses in the end of this section: http://openid.net/specs/openid-authentication-2_0.html#direct_comm and 8.2.4. Unsuccessful Response Parameters in the end of this section: http://openid.net/specs/openid-authentication-2_0.html#anchor20). When I tested Livejournal with cl-openid last time, it worked OK.

Update: Livejournal problem is solved by workaround in cl-openid. Blogger remains broken (including work with any other OpenID relying parties, not only cl-openid).

Monday, 24 January 2011

В C++ вводят лямбда функции и замыкания.

Я практически не пользуюсь С++ уже несколько лет, и не мог предположить, что благодать нисходит на него так интенсивно. Изменения языка довольно существенные. Сборщик мусора правда не вводят. Конечно, существующие коллекторы [2] никто не отменял, но в стандарт не включают.
1 - https://secure.wikimedia.org/wikipedia/ru/wiki/C%2B%2B0x
2 - https://secure.wikimedia.org/wikipedia/en/wiki/Boehm_garbage_collector

Wednesday, 2 June 2010

Указ 60 .net


Актуально для белорусов: http://ukaz60.net
Статистика за время с первой подписи 28 мая 10:49 по текущий момент 2 часа ночи 3 апреля. Число голосов в опросе 3301. Число "подписей" к письму в виде комментариев - 1806.
Возможно есть и другие подобные сайты, этот я нашел поиском в гугле. Нужно будет подумать, что еще можно сделать против этого вредного указа.

Monday, 3 May 2010

Прочитал на тут бае:

Апостол Павел считается небесным покровителем органов финансовых расследований Республики Беларусь.

http://news.tut.by/society/168946.html

Friday, 16 October 2009

Saturday, 22 August 2009

Extending IntelliJ IDEA using Common Lisp

If you have nothing to do, you can embed ABCL into IntelliJ IDEA and extend and control IDEA interactively from SLIME.

I created a little plugin that starts ABCL inside of IDEA:


Note: both videos for better resolution are opened in new window at their Vimeo URLs

How it works: "ABCL Start" command creates ABCL compiler and attaches it's REPL to a special "ABCL Console" tool window. Also it executes start.lisp file, which is supposed to be edited by user. User may do whatever he needs: start SLIME server or load other lisp files.

SLIME fuzzy-completion works for Java classes/methods because I use jfli (I found an ABCL port by Andras Simon). I wish fuzzy-completion work faster with ABCL (the latency may be up to half a second). But I believe ABCL will be improved in this area in the near future.

Hello Word -like examples in the video above are contained in the fun.lisp file from the plugin distribution.

Another example I prepared is implementation of Emacs-like "yanking" in IDEA (Ctrl-Y pastes an item from clipboard, and Alt-Y replaces just inserted item by next clipboard item). The code is contained in the yanking.lisp. Here it is in action:


To summarize, it is possible, and with some practice quite convenient, to use ABCL for IDEA extension.

If you have in mind some useful extension that may be implemented in Lisp, you can use my code as a boilerplate.

The sources: http://github.com/avodonosov/abcl-idea/
Binary distro: http://github.com/avodonosov/abcl-idea/downloads

Some details are provided in the README file.

Saturday, 13 December 2008

Винни-Пых

Читал сейчас в википедии статью про Винни-Пуха. В частности, там рассказывается про звучание имени на разных языках. Например, в английском языке «h» в имени Pooh не произносится, это имя рифмуется постоянно с who или do.

В белорусском переводе некоего Виталя Воронова — Віня-Пых. Вторая часть имени переведена как «Пых», якобы потому, что это созвучно с белорусскими словами пыха (высокомерие и гордость) и запыхацца.

Sunday, 7 December 2008

Как скрыть рекламу в Live Journal

Я читаю несклько блогов в Live Journal и мне мешает громадный рекламный баннер на пол экрана, который Live Journal вставляет в блоги бесплатных аккаунтов.

AdBlock плагин к Firefox в данном случае не помог, он скрывает картинку, но тогда остается большое белое пространство, что не намного лучше картнинки.

В Google быстро нашлось решение.

Для Firefox

Создаем файл userContent.css:


/* Removes LJ Ads from pages. */
/* Code From ambience8 */

@-moz-document domain("livejournal.com") {

div.adv {display: none !important; visibility: hidden !important;}
div.ljad {display: none !important; visibility: hidden !important;}
div.ljadleaderboard-bottom {display: none !important; visibility: hidden !important;}
div.ljadleaderboard {display: none !important; visibility: hidden !important;}
div.ljadwrapper-journal-after-post-c {display: none !important; visibility: hidden !important;}
div.ljadwrapper-app-home {display: none !important; visibility: hidden !important;}
div.ljadmedrect {display: none !important; visibility: hidden !important;}

}
Этот файл нужно положить в директорию chrome вашего профиля. На Widows XP это будет C:\Documents and Settings\<Имя Пользователя>\Application Data\Mozilla\Firefox\Profiles\xxxxxxxx.default\chrome\userContent.css. В Windows Vista: C:\Users\<Имя Пользователя>\AppData\Roaming\Mozilla\Firefox\Profiles\xxxxxxxx.default\chrome\userContent.css

Теперь перезапускаем Firefox. В результате реклама в блогах Live Journal должна исчезнуть.

Для Internet Explorer

Создаем файл userContent.css:

/* Removes LJ Ads from pages. */
/* Code From ambience8 */

div.adv {display: none !important; visibility: hidden !important;}
div.ljad {display: none !important; visibility: hidden !important;}
div.ljadleaderboard-bottom {display: none !important; visibility: hidden !important;}
div.ljadleaderboard {display: none !important; visibility: hidden !important;}
div.ljadwrapper-journal-after-post-c {display: none !important; visibility: hidden !important;}
div.ljadwrapper-app-home {display: none !important; visibility: hidden !important;}
div.ljadmedrect {display: none !important; visibility: hidden !important;}

В меню Internet Explorer выбираем Tools -> Internet Options -> General -> Accessibility -> User style sheet. Здесь указываем путь к созданному файлу.

Как можно заметить из кода, решение для Internet Explorer будет скрывать элемены div с классом adv на любых страницах, а не только в livejournal.com, так что... это не страшно :)

Источник: http://wiki.noljads.com/Hiding_LiveJournal_ads_with_user_stylesheets

Friday, 5 December 2008

Hunchentoot on ECL

It was reported that all Hunchentoot dependencies are already ported to ECL and a patch exists for Hunchentoot itself to make it running on ECL.

I am glad to hear this, because ECL is the only free Common Lisp on Windows with multithreading support (at the same time an active work on multithreading for CLISP is in progress, Clozure CL has beta port for Windows, and a lot of work was also done for threading in SBCL Windows port, although the work is not finished).

I tried Hunchentoot on ECL and for those who are interested too, I reproduce here the steps I took. My environment: Windows XP + Microsoft Visual Studio.

  1. You need the latest versions of Hunchentoot and all the dependencies. If you are lazy, you may get all them at once from this archive I prepared.
  2. Apply the patch from this post (already done in my archive): save the port-ecl.lisp to the hunchentoot directory and add #+:ecl (:file "port-ecl") to hunchentoot.asd as here.
  3. Checkout fresh ECL from CVS:
      cvs -z3 -d:pserver:anonymous@ecls.cvs.sourceforge.net:/cvsroot/ecls checkout ecl
    
    I recommend fresh checkout because I had problems building ECL from sources already existing on my computer after update with cvs update -dAP.
  4. Configure ECL. Open ecl\msvc\Makefile and set
    ECL_THREADS  = 1
    ECL_UNICODE  = 1
    ECL_ASDF     = 1
    ECL_SOCKETS  = 1
    
  5. Build ECL. Run Visual Studio command prompt: menu Start->All Programs -> Microsoft Visual Studio 2005 -> Visual Studio Tools -> Visual Studio 2005 Command Prompt. This script sets up environment variables so that C compiler cl.exe, nmake.exe and other things are in your PATH, all includes and libraries are available, etc. In this console window type:
     > cd <DIR_WITH_ECL_SOURCES>\ecl\msvc
     > nmake
    
    After the build completes, copy resulting binaries somewhere:
      > nmake install prefix=c:\somewhere\ecl
    Now you have working ECL, lets test it:
    > c:\somewhere\ecl\ecl.exe
    ECL (Embeddable Common-Lisp) 0.9l (CVS 2008-07-12 18:54)
    Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
    Copyright (C) 1993 Giuseppe Attardi
    Copyright (C) 2000 Juan J. Garcia-Ripoll
    ECL is free software, and you are welcome to redistribute it
    under certain conditions; see file 'Copyright' for details.
    Type :h for Help.  Top level.
    > (+ 2 2)
    4
    > (quit)
    
    Works.

    Do not close the Visual Studio command prompt, we will need it soon.

  6. As Hunchentoot uses ASDF, we need Hunchentoot with all its dependencies to be registered in ASDF:*CENTRAL-REGISTY*. For this I have a special lisp file, setup-asdf-registry.lisp (already included in the archive from the step 1):
    ;; this file must be stored in the same directory
    ;; where all the lisp libraries are stored 
    ;; (but libraries, of course, have their own subdirectories)
    
    (require 'asdf)
    
    (flet
        ((reg (relative-lib-dir)
           (let ((lib-dir
                  (directory-namestring 
                   (merge-pathnames relative-lib-dir
                                    (load-time-value
                                     (or #.*compile-file-pathname* 
                                         *load-pathname*))))))
             (pushnew lib-dir
                      asdf:*central-registry*
                      :test #'equalp))))
      (reg "alexandria\\")
      (reg "asdf-binary-locations\\")
      (reg "babel_0.2.0\\")
      (reg "cffi-080926\\")
      (reg "chunga-0.4.3\\")
      (reg "cl-base64-3.3.2\\")
      (reg "cl-fad-0.6.2\\") 
      (reg "cl-ppcre-2.0.1\\")
      (reg "cl-who-0.11.0\\")
      (reg "cl+ssl\\")
      (reg "flexi-streams-1.0.7\\")
      (reg "hunchentoot-0.15.7\\")
      (reg "md5-1.8.5\\")
      (reg "rfc2388\\")
      (reg "trivial-features_0.1\\")
      (reg "trivial-gray-streams\\")
      (reg "url-rewrite-0.1.1\\"))
    
    (asdf:operate 'asdf:load-op :asdf-binary-locations) 
  7. Compile and load Hunchentoot. For this go back to the Visual Studio command prompt.
    > c:\somewhere\ecl\ecl.exe  
    > (load "c:/archive-from-the-step-1/setup-asdf-registry.lisp") 
    > (pushnew :hunchentoot-no-ssl *features*)
    > (asdf:operate 'asdf:load-op :hunchentoot-test)
    
    ECL must be started from Visual Studio command prompt because it uses C compiler when compiling Lisp. (pushnew :hunchentoot-no-ssl *features*) is only necessary if you do not have OpenSSL installed (get it here for Win32).
  8. Start Hunchentoot:
    > (hunchentoot:start-server :port 4242) 
    

Now you may open http://localhost:4242/hunchentoot/test and see Hunchentoot test site.

At the moment of this post writing, almost all examples from the test site work, except for two: "UTF-8 demo (writing UTF-8 characters directly to the stream)" and "UTF-8 demo (returning a string)". The reason is already found and I am sure it will be fixed very soon.

Wednesday, 3 December 2008

Simpler Clojure/SLIME Embedding

Thanks to Craig McDaniel's hint, I know a simpler way to embed Clojure/SLIME than I used before. Since Clojure svn revision 1127, macro clojure.main/with-bindings makes the embedding as trivial as this jsp page.

Friday, 28 November 2008

Clojure at Work: Interactively Control Running Java System

We are developing a system on Java. It must call another system and at some point the other system calls our system back via web service. Therefore I can not test this interaction in a stand-alone test, our code must be executed in an HTTP container (we use Tomcat).

In case if the code does not work as desired, I need to stop Tomcat, fix and rebuild Java sources and try again. And what is worse, before the code I am working on can be tested, it performs initialization that takes about a minute. These stumbles are little bit annoying for those who are used to fully dynamic development environments as Lisp.

I decided to embed Clojure into the system during development and connect to it with SLIME to create interactive testing/development environment.

First, I followed Bill Clementson's instructions for usual (not embedded) Clojure/SLIME setup.

After it worked, I started to look for a way to embed Clojure/swank into existing Java system, but didn't find any conventional way to do this. Therefore I torn out a piece of code from the clojure.lang.Repl class and instead of standard input, feed it with commands to start swank (which I found in the *inferior-lisp* buffer when Clojure/SLIME is started "normally", from Emacs).

All this is packed into simple clojure.jsp page in my web application directory. And clojure.jar is placed into WEB-INF/lib.

When the JSP is opened in browser it displays: "swank has been stared".

After that, I use M-X slime-connect from Emacs. The port number, as you can see from the code, is 7777.

What is the result?

I can inspect and control the running system, for example invoke functions I need to test or change object states.

But I must admit that the level of flexibility is lower than I hoped for. The point is that I can not interactively change Java code (which constitutes all the system). To be more flexible, the system must be written on Lisp entirely.

Links

Thursday, 23 October 2008

Friday, 29 August 2008

Poor Man's Anti-Virus или Doctor Web на халяву

Что делать, если нужно проветить файл на наличие вирусов, а покупать лицензионный атнивирус душит жаба; пользоваться крэкнутым не позволяет совесть или то, что при обновлении баз он перестает работать?

Мысль развивалась так:

А что, если купить один лицензионный и поставить в локалке на всю организацию или в инете сервак такой завести: сабмитишь файл и тебе говорят, есть там вирусы или нет.

Потом дошло, такое уже есть, причем совершенно бесплатно. Большинство бесплатных почтовых служб, например gmail и yandex, проверяют всю полученную почту на вирусы.

Т.е. чтобы проверить файл на вирусы, отправляешь его себе по почте. Если вирусы есть - в пришедшем письме будет предупреждающее сообщение.

Gmail в этом смысле неудобен (во всяк. случае для отправки). Потому, что отправить письмо с файлом .exe не разрешает, и даже не разрешает отправлять архивы содержащие .exe внутри.

А yandex позволяет отправлять .exe. Когда письмо придет, там даже есть такая удобная кнопочка "Проверить на вирусы", чтобы явно увидеть ответ: "Проверено антивирусом Doctor Web. Вирусов не обнаружено!".

Thursday, 14 August 2008

Классно почитал Тура Хейердала

Блуждая в википедии увидел Тура Хейердала и узнал, что он писал книги о своих путешествиях. Открыл для любопытства здесь: http://www.lib.ru/ALPINISM/HEJRDAL/heierdal_kontiki.txt, начал со случайного места. И прочитал целиком (не за один день; кажется, дня три подряд читал: на выходных и помню будний день тоже был: прихожу с работы и сажусь читать).

Давно не читал художественную литературу, что-то неинтересно, не получаю удовольствия. Эта, правда, не художественная (в том смысле, что описывает реальные события). Может потому и понравилась.

Saturday, 9 August 2008

О чем думали люди 4-5 тысяч лет назад

Эпос о Гильгамеше: http://www.russianplanet.ru/filolog/epos/gilgamesh/text.htm.
Пересказ сюжета и историю можно найти на этом же сайте или в википедии.

Tuesday, 15 July 2008

GUI для SMS банкинга

http://smsbanking.googlepages.com/ - удобная программа для пользователей карточек Беларусбанка.

Репозитарий подверсий

http://www.livejournal.com/code/. Если язык интерфейса русский, то на этой странице
можно прочитать: "Вы можете получить код сервера Живого Журнала из репозитария подверсий."

В английском варианте - "the Subversion repository".

Monday, 7 July 2008

Good News

Tetris in emacs is customizable, so I can play starting right at the interesting speed :)

Here is my value for the tetris-update-speed-function:
(lambda (shapes rows) (/ 20.0 (+ 180.0 rows))

Monday, 30 June 2008

Если любишь ты футбол - 50 за каждый гол!

Самый забавный из 5-6 рекламных роликов водки, которые показывали в перерыве между таймами футбольного полуфинала.

Совсем офигели, бухло рекламировать. Уже почти что в рот тебе его наливают. "50 за маму, 50 за папу, 50 за дедушку, 50 за бабушку..."

Thursday, 26 June 2008

Fix Oracle Output

Customers recently sent us for analysis results of several queries from their Oracle database. The output produced by SQL*Plus or TOAD with the default value of the LINESIZE system variable is quite nasty. Similar to the following:

SQL> select * from users;

        ID
----------
NAME                                                                            
--------------------------------------------------------------------------------
SURNAME                                                                         
--------------------------------------------------------------------------------
LOGIN                                                                           
--------------------------------------------------------------------------------
PASSWORD                                                                        
--------------------------------------------------------------------------------
REGISTRATION_TIME            
-----------------------------
         1
ivan                                                                            
ivanov                                                                          
vanya                                                                           
123                                                                             
26.06.08 21:16:12,000000     
                                                                                
         2
petr                                                                            
petrov                                                                          
petya                                                                           

        ID
----------
NAME                                                                            
--------------------------------------------------------------------------------
SURNAME                                                                         
--------------------------------------------------------------------------------
LOGIN                                                                           
--------------------------------------------------------------------------------
PASSWORD                                                                        
--------------------------------------------------------------------------------
REGISTRATION_TIME            
-----------------------------
qwerty                                                                          
26.06.08 21:16:12,000000     
                                                                                
         3
sidor                                                                           
sidorov                                                                         
sid                                                                             
password                                                                        
26.06.08 21:16:12,000000     
                                                                                


3 rows selected.

SQL> spool off;

 
I created a little tool to convert such an output to a pretty thing like this:
ID         NAME                                                                             SURNAME                                                                          LOGIN                                                                            PASSWORD                                                                         REGISTRATION_TIME             
---------- -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- ----------------------------- 
         1 ivan                                                                             ivanov                                                                           vanya                                                                            123                                                                              26.06.08 21:16:12,000000      
         2 petr                                                                             petrov                                                                           petya                                                                            qwerty                                                                           26.06.08 21:16:12,000000      
         3 sidor                                                                            sidorov                                                                          sid                                                                              password                                                                         26.06.08 21:16:12,000000      

Lisp source is here. Usage instructions are at the top of the file.

Monday, 28 April 2008

Learning Flex

I'm learning Adobe Flex now. Surprisingly, I have much better impression than I expected about it (my expectations are derived from one or two evening experience with Flash).
I dislike ajax/comet etc, because IMHO web browsers are not suitable for desktop-like UI and it is high time for a new cross-platform client technology.
Flex Builder resembles old good Borland Delphi or MS VisualStudio. Flex libraries and tools have "open design" (my intuitive definition for this: if there is no standard TreeView class, we could write it ourselves because all base classes/interactions are understandably defined and the sources are available).
BTW, interesting facts about the language. Adobe claims that ActionScript is based on ECMAScript Edition 4. The ECMAScript 4 language is in some sense progressive, has interesting (in particular lisp-like) features: http://www.ecmascript.org/es4/spec/overview.pdf. Also funny is that the reference implementation of ECMAScript is created on Standard ML.

Thursday, 6 March 2008

Junit 4.4 new assertion mechanism is a very clear demonstration of java limitation comparing to lisp.

It is obvious that ASSERT A > 2 OR B < 3 when failed should print a message "assertion failed: a > 2 or b < 3. This is trivial to implement with lisp macros, but when the language does not provide programmatic access to the program source code, our ASSERT receives only result of expression calculation - a boolean.

Therefore people must create new expression specification language and duplicate expression evaluation that is already present in java compiler.

Thursday, 7 February 2008

Interesting lisp (or scheme) project

There was a post in the lispjobs blog - a part time job for a freelance lisp programmer (I even applied).

The project in question is quite interesting - a website where open source software may be funded collectively by the software users.

Also interesting is that the project is going to be open source and to be created on lisp (they already started and use PLT Scheme).

The guy running this project - Christopher Rasch - wrote quite detailed article almost seven years ago about how such a system may work.

In two words it is approximately following.

Some people wish some soft to be created and are ready to pay some money for it. These people may be called "donors". Other people may do the work required. They may be called "performers". The goal is to allow donors to collect in common sufficient money for performer(s) to do the work. After that performer(s) do the work and get the money.

The mechanism suggested to achieve this goal is similar to equity market. Central object is obligation to do the required work - it is called "software completion bond". The bond is "backed" by money of donors - this means that bond owner may exchange the bond to the money when the job completion criteria are met (as determined by the judge indicated in the work specification).

Bond owner is the performer - he acquires the bonds by some small price before starting work. He may even do only part of the work and sell the bonds to someone else. It is supposed that in this case the bond price will be higher than the initial because less work is left to do to receive the money escrowed for this job.

Christopher Rasch describes his project as a "wish trading market". Read the article for more details.

Although implementation may vary, there are people that want something in this fashion. See at least this thread in the sbcl-devel mailing list.

As for me, I think it is a very interesting idea. There are several projects that I would fund by few dollars, especially taking into account that the money are protected until the job is done. And in particular the "wish trading market"- I hope a kind of bootstrapping may be applied here: after some initial version, this project development may be funded via the described mechanism too.

How comprehensive this initial version should be? Maybe most of things may be done manually at the beginning.

I hope the project will succeed. Unfortunately there is not so much progress last time (AFAICS there were no changes at all in the project svn repository for at least several last weeks).

Links

Tuesday, 1 January 2008

Web development in single threaded lisp with slime and hunchentoot

It is problematic at first sight to use SLIME for web development in a single threaded lisp with swank:*communication-style* = nil (as it happens with CLISP and SBCL on Windows). The problem is the impossibility to modify application while it is running.

In very informal pseudo code this configuration of swank may be simplified represented as:

(loop
   (read-slime-request)
   (exec-slime-request))
When our next SLIME request starts hunchentoot, hunchentoot enters its own request handling loop (for HTTP requests):
(loop
   (read-slime-request)
   (exec-slime-request :
      (loop
         (read-http-request)
         (handle-http-request)))
This means exec-slime-request will not exit and subsequent slime requests will not be handled until the web server is stopped. SLIME isn't functioning.

The solution is to use web server to invoke SLIME. When in the emacs we are going to talk to swank, send special HTTP request and its handler will enforce swank to handle emacs request.

(loop
   (read-slime-request)
   (exec-slime-request :
      (loop
         (read-http-request)
         (handle-http-request :
            (read-slime-request)
            (exec-slime-request))))
It requires only few lines of code in emacs lisp and in common lisp. The code is here, it's simple, see it for more details.

One note is needed to help fully understand the code. When emacs side sends a request to swank, swank is not always blocked by hunchentoot. As far as I understand it, one slime request may lead to a sequence of communications between swank and emacs.

01: (loop
02:   (read-slime-request)
03:   (exec-slime-request :
04:      (loop
05:         (read-http-request)
06:         (handle-http-request :
07:            (read-slime-request)
08:            (exec-slime-request :
09:               ...
10:               (read-slime-request)
11:               ...))))
In the emacs we should not make an HTTP request before sending the slime request that is to be read on the line 10.

To distinguish this situation on the emacs side I check whether the socket of HTTP request read by swank on the line 05 is still connected. In this case new HTTP request isn't performed.

This solution isn't 100% correct (the socket may be still not closed, but swank isn't going to read anything). But it works almost good. Sometimes a glitch appears - when you execute an expression, you get result of previous expression instead of the just executed. If you execute expressions en, en+1, en+2, you get results of executing expressions en-1, en, en+1. This "phase displacement" is most likely caused by the fact that we not performed HTTP request when it was necessary and the slime request remained not read by swank. When next time we do HTTP request for the next slime request, the previous unread request is read and executed by swank.

In principle, I know how to fix this problem (we may annotate HTTP requests and slime requests with some IDs and do synchronization - ignore unnecessary HTTP requests - on the common lisp side; socket checking on the emacs side will became unnecessary). But I do not want to spend time on it, it is easier to enforce reading enqueued expressions by the "special HTTP request" directly from browser: fetch http://localhost:4343/slime-http-send-handler needed count of times.

How to use it.

Very easy.

Add the following line into your .emacs file:

(load "c:/some/path/slime-over-http.el")
The line should follow the normal slime setup.

Here is an example of common lisp web site:

;; this line may vary in you case; just make
;; sure that asdf is loaded and all the needed
;; libraries are available
(load "c:/usr/unpacked/lisp-libs/setup-asdf-registry.lisp")

(asdf:operate 'asdf:load-op :hunchentoot)
(use-package :hunchentoot)

(defun want-slime-over-http-p ()
  "This predicate indicates whether we
are in the development environment and have no
threading"
  #+:win32 t)

(when (want-slime-over-http-p)
  (load "C:/usr/projects/slime-over-http/slime-over-http.lisp")
  (defun hunchentoot::keep-alive-p ()))

(setq *dispatch-table*
      `(dispatch-easy-handlers
        ,@(when (want-slime-over-http-p)
                (list (create-prefix-dispatcher
                           "/slime-http-send-handler"
                           (find-symbol "SLIME-HTTP-SEND-HANDLER"
                                        "SLIME-HTTP"))))
        default-dispatcher))

(define-easy-handler (hello :uri "/hello")
    ()
  "Hello, change me!")
After executing the above code run emacs command slime-start-hunchentoot. Now you may open http://localhost:4343/hello in the browser and see the phrase "Hello, change me!".

You may change it because you have fully functioning slime with repl, debugger, etc.

To stop the server execute common lisp function (slime-http::stop-instance).

Note how we redefined hunchentoot::keep-alive-p to return nil. Single threaded server can't provide keep-alive, because while it is handling alive connection it can't accept any other connections (in particular from slime; also, IE is often unable to fully open a page with several images or frames in this case; looks like it opens several connections and schedules downloading different parts of the page to different connections; connections other than "kept alive" are not handled by hunchentoot).

In addition to redefining keep-alive-p I also use reverse proxying apache for sure.

Resume

So we see quite simple way (despite the long explanation) to do web development with hunchentoot in free lisps available on windows.

It isn't the only way to develop web applications with common lisp if you are on windows. For example you may install linux virtual machine. Edi Weitz - the hunchentoot author - uses this approach.

PS

My code is called "slime-over-http" because initially I delivered slime requests over the TCP connection used by the HTTP request and only some time later I realized that it is unnecessary. I'd like another name (preferably short), but can't contrive it.

If you have any questions about this configuration - ask, I'll be glad to help.

Wednesday, 12 December 2007

ECL - multi threaded, free lisp, supporting Windows

Until yesterday I thought that there are no free multithreaded Common Lisp implementations for Windows.

But it turned out that ECL supports multithreading.

Yesterday I downloaded, build it and tried simple test: (mp:process-run-function 'zu (lambda () (loop (format t "zu~%") (sleep 3))))

Seems working.

Also ECL may be build with Unicode support. Although it doesn't support external formats in streams, strings and characters are represented as 24-bit values. It should be sufficient to have support for various external formats in IO using flexi-streams.

Good news for me. I pin great hopes on ECL.

P.S.

I discovered ECL via Common Lisp Survey linked from the xach's post. Thanks ... all.