lua.xml
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
<!ENTITY % darktable_dtd SYSTEM "../dtd/darktable.dtd">
%darktable_dtd;
]>
<chapter status="final" id="lua_chapter">
<chapterinfo>
<keywordset>
<keyword>scripting</keyword>
<keyword>lua</keyword>
</keywordset>
</chapterinfo>
<title>Scripting with Lua</title>
<indexterm>
<primary>Lua</primary>
</indexterm>
<indexterm>
<primary>scripting</primary>
</indexterm>
<para>
darktable comes with a versatile scripting interface for functionality enhancement.
</para>
<para>
<screenshot>
<screeninfo>Scipting with Lua</screeninfo>
<graphic fileref="images/lua_view.png" scalefit="1" width="80%"></graphic>
</screenshot>
</para>
<sect1 id="lua_usage">
<title>Lua usage</title>
<para>
Lua can be used to define actions which darktable will perform whenever a specified event
is triggered. One example might be calling an external application during file export in
order to apply additional processing steps outside of darktable.
</para>
<para>
darktable uses <ulink url="http://www.lua.org/"><emphasis>Lua</emphasis></ulink>, which is
an independent project founded in 1993, providing a powerful, fast, lightweight,
embeddable scripting language. Lua is widely used by many open source applications, in
commercial programs, and for games programming.
</para>
<para>
darktable uses Lua version 5.2. Describing the principles and syntax of Lua is beyond the
scope of this usermanual. For a detailed introduction see the
<ulink url="http://www.lua.org/manual/5.2/manual.html">Lua reference manual</ulink>.
</para>
<sect2 id="lua_basic_principles">
<title>Basic principles</title>
<para>
At startup, darktable will automatically run two Lua scripts:
</para>
<itemizedlist>
<listitem><para>
a script called <emphasis>luarc</emphasis> in
<emphasis>$DARKTABLE/share/darktable</emphasis>
</para></listitem>
<listitem><para>
a script called <emphasis>luarc</emphasis> in the user's configuration directory
</para></listitem>
</itemizedlist>
<para>
$DARKTABLE is used here to represent your system's darktable installation directory.
</para>
<para>
This is the only time darktable will run Lua scripts by itself. The script can register
callbacks to perform actions on various darktable events. This callback mechanism is the
primary way to trigger lua actions.
</para>
</sect2>
<sect2 id="lua_hello_world">
<title>A simple lua example</title>
<para>
Let's start with a simple example. We will print some code on the console. Create a file
called <emphasis>luarc</emphasis> in darktable's configuration directory (usually
<command>~/.config/darktable/</command>) and add the following line to it:
</para>
<para>
<programlisting language="lua">print("Hello World !")</programlisting>
</para>
<para>
Start darktable and you will see the sencence <emphasis>Hello World !</emphasis> printed
on the console. Nothing fancy but it's a start...
</para>
<para>
At this point, there is nothing specific to darktable in the script. We simply use the
standard
<code>print</code>
function to print a string. That's nice and all, but we can do better than that. To
access the darktable API you first need to
<code>require</code>
it and save the returned object in a variable. Once this is done you can access the
darktable API as subfields of the returned object. All of this is documented in
darktable's Lua API reference manual (see <xref linkend="lua_api"/>).
</para>
<para>
<programlisting language="lua">local darktable = require "darktable"
darktable.print_error("Hello World !")</programlisting>
</para>
<para>
Run the script ... and nothing happens. The function
<code>darktable.print_error</code>
is just like print but will only print the message if you have enabled lua traces with
<command>-d lua</command> on the command line. This is the recommended way to do traces
in a darktable lua script.
</para>
</sect2>
<sect2 id="lua_print_images">
<title>Printing labeled images</title>
<para>
This first example showed us the very basics of lua and allowed us to check that
everything is working properly. Let's do something a little bit more complex. Let's try
to print the list of images that have a red label attached to them. But first of all,
what is an image?
</para>
<para>
<programlisting language="lua">
local darktable = require "darktable"
local debug = require "darktable.debug"
print(darktable.debug.dump(darktable.database[1]))</programlisting>
</para>
<para>
Running the code above will produce a lot of output. We will look at it in a moment, but
first let's look at the code itself.
</para>
<para>
We know about requiring
<code>darktable</code>
. Here, we need to separately require
<code>darktable.debug</code>
which is an optional section of the API that provides helper functions to help debug lua
scripts.
</para>
<para>
<code>darktable.database</code>
is a table provided by the API that contains all images in the database (currently
visible or not, duplicate or not...). Each entry in the database is an image object.
Image objects are complex objects that allow you to manipulate your image in various
ways (it is all documented in the
<code>types_dt_lua_image_t</code>
section of the API manual). To display our images, we use
<code>darktable.debug.dump</code>
which is a function that will take anything as its parameter and recursively dump its
content. Since images are complex objects that indirectly reference other complex
objects, the resulting output is huge. Below is a cut down example of the output.
</para>
<para>
<literallayout>
<code>
toplevel (userdata,dt_lua_image_t) : /images/100.JPG
publisher (string) : ""
path (string) : "/images"
move (function)
exif_aperture (number) : 2.7999999523163
rights (string) : ""
make_group_leader (function)
exif_crop (number) : 0
duplicate_index (number) : 0
is_raw (boolean) : false
exif_iso (number) : 200
is_ldr (boolean) : true
rating (number) : 1
description (string) : ""
red (boolean) : false
get_tags (function)
duplicate (function)
creator (string) : ""
latitude (nil)
blue (boolean) : false
exif_datetime_taken (string) : "2014:04:27 14:10:27"
exif_maker (string) : "Panasonic"
drop_cache (function)
title (string) : ""
reset (function)
create_style (function)
apply_style (function)
film (userdata,dt_lua_film_t) : /images
1 (userdata,dt_lua_image_t): .toplevel
[......]
exif_exposure (number) : 0.0062500000931323
exif_lens (string) : ""
detach_tag (function): toplevel.film.2.detach_tag
exif_focal_length (number) : 4.5
get_group_members (function): toplevel.film.2.get_group_members
id (number) : 1
group_with (function): toplevel.film.2.group_with
delete (function): toplevel.film.2.delete
purple (boolean) : false
is_hdr (boolean) : false
exif_model (string) : "DMC-FZ200"
green (boolean) : false
yellow (boolean) : false
longitude (nil)
filename (string) : "100.JPG"
width (number) : 945
attach_tag (function): toplevel.film.2.attach_tag
exif_focus_distance (number) : 0
height (number) : 648
local_copy (boolean) : false
copy (function): toplevel.film.2.copy
group_leader (userdata,dt_lua_image_t): .toplevel
</code>
</literallayout>
</para>
<para>
As we can see, an image has a large number of fields which provide all sort of
information about it. We are interested in the red label. This field is a boolean, and
the documentation tells us that it can be written. We now just need to find all images
with that field and print them out.
</para>
<para>
<programlisting language="lua">
darktable = require "darktable"
for _,v in ipairs(darktable.database) do
if v.red then
print(tostring(v))
end
end</programlisting>
</para>
<para>
This code should be quite simple to understand at this point, but it contains a few
interesting aspects about lua that are worth highlighting:
<itemizedlist>
<listitem><para>
<code>ipairs</code>
is a standard lua function that will iterate through all numeric indices of a table.
We use it here because
<code>darktable.database</code>
has non-numeric indices which are functions to manipulate the database itself
(adding or deleting images, for example).
</para></listitem>
<listitem><para>
Iterating through a table will return both the key and the value used. It is
conventional in lua to use a variable named <quote>
<code>_</code>
</quote> to store values that we don't care about.
</para></listitem>
<listitem><para>
Note that we use the standard lua function
<code>tostring</code>
here and not the darktable specific
<code>darktable.debug.dump</code>
. The standard function will return a name for the object whereas the debug function
will print the content. The debug function would be too verbose here. Once again, it
is a great debug tool but should not be used for anything else.
</para></listitem>
</itemizedlist>
</para>
</sect2>
<sect2 id="lua_simple_shortcut">
<title>Adding a simple shortcut</title>
<para>
So far, all our scripts have done things during startup. This is of limited use and
doesn't allow us to react to real user actions. To do more advanced things we need to
register a function that will be called on a given event. The most common event to react
to is a keyboard shortcut.
</para>
<para>
<programlisting language="lua">
darktable = require "darktable"
local function hello_shortcut(event, shortcut)
darktable.print("Hello, I just received '"..event..
"' with parameter '"..shortcut.."'")
end
darktable.register_event("shortcut",hello_shortcut,
"A shortcut that print its parameters")
</programlisting>
</para>
<para>
Now start darktable, go to
<code> preferences => shortcut => lua => A shortcut that print its parameters </code>
assign a shortcut and try it. You should have a nice message printed on the screen.
</para>
<para>
Let's look at the code in details. We first define a function with two parameters. These
parameters are strings. The first one is the type of event that is triggered (
<code>"shortcut"</code>
) and the second one is what shortcut specifically (
<code>"A shortcut that print its parameters"</code>
). The function itself calls
<code>darktable.print</code>
that will print the message as an overlay in the main window.
</para>
<para>
Once that function is defined, we register it as a shortcut callback. To do that we call
<code>darktable.register_event</code>
which is a generic function for all types of events. We tell it that we are registering
a shortcut event, then we give the callback to call and last, we give the string to use
to describe the shortcut in the preference window.
</para>
<para>
Let's try a shortcut a little more interactive. This one will look at the images the
user is currently interested in (selected or moused-over) and will increase their
rating.
</para>
<para>
<programlisting language="lua">
darktable = require "darktable"
darktable.register_event("shortcut",function(event,shortcut)
local images = darktable.gui.action_images
for _,v in pairs(images) do
v.rating = v.rating + 1
end
end,"Increase the rating of an image")
</programlisting>
</para>
<para>
At this point, most of this code should be self explanatory. Just a couple of notes:
<itemizedlist>
<listitem><para>
Instead of declaring a function and referencing it, we declare it directly in the
call to
<code>darktable.register_event</code>
this is strictly equivalent but slightly more compact.
</para></listitem>
<listitem><para>
<code>image.rating</code>
is a field of any image that gives its rating (between 0 and 5 stars, -1 means
rejected).
</para></listitem>
<listitem><para>
<code>darktable.gui.action_images</code>
is a table containing all the images of interest. darktable will act on selected
images if any image is selected, and on the image under the mouse if no image is
selected. This function allows to easily follow darktable's UI logic in lua.
</para></listitem>
</itemizedlist>
</para>
<para>
If you select an image and press your shortcut a couple of time, it will work correctly
at first but when you have reached five stars, darktable will start showing the
following error on the console:
</para>
<para>
<literallayout>
<code>
<![CDATA[
LUA ERROR : rating too high : 6
stack traceback:
[C]: in ?
[C]: in function '__newindex'
./configdir/luarc:10: in function <./configdir/luarc:7>
LUA ERROR : rating too high : 6
]]>
</code>
</literallayout>
</para>
<para>
This is lua's way of reporting errors. We have attempted to set a rating of 6 to an
image, but a rating can only go as high as 5. It would be trivial to add a check, but
let's go the complicated way and catch the error instead.
</para>
<para>
<programlisting language="lua">
darktable.register_event("shortcut",function(event,shortcut)
local images = darktable.gui.action_images
for _,v in pairs(images) do
result,message = pcall(function()
v.rating = v.rating + 1
end)
if not result then
darktable.print_error("could not increase rating of image "..
tostring(v).." : "..message)
end
end
end,"Increase the rating of an image")</programlisting>
</para>
<para>
<code>pcall</code>
will run its first argument and catch any exception thrown by it. If there is no
exception it will return
<code>true</code>
plus any result returned by the function; if there is an exception it will return
<code>false</code>
and the error message of the exception. We simply test these results and print them on
the console...
</para>
</sect2>
<sect2 id="lua_storage_example">
<title>Exporting images with Lua</title>
<para>
We have learned to use lua to adapt darktable to our particular workflow, let's look at
how to use lua to easily export images. darktable can easily export images to quite a
few online services but there are always more. If you are able to upload an image to a
service via the command line then you can use lua to integrate it into darktable's user
interface.
</para>
<para>
In this next example we will use lua to export via <command>scp</command>. A new storage
will appear in darktable's UI which will export images to a remote target via ssh's copy
mechanism.
</para>
<para>
<programlisting language="lua">
darktable = require "darktable"
darktable.preferences.register("scp_export","export_path",
"string","target SCP path",
"Complete path to copy to. Can include user and hostname","")
darktable.register_storage("scp_export","Export via scp",
function( storage, image, format, filename,
number, total, high_quality, extra_data)
if coroutine.yield("RUN_COMMAND","scp "..filename.." "..
darktable.preferences.read("scp_export",
"export_path","string")) then
darktable.print_error("scp failed for "..tostring(image))
end
end)</programlisting>
</para>
<para>
<code>darktable.preferences.register</code>
will add a new preference to darktable's preference menu.
<code>scp_export</code>
and
<code>export_path</code>
allow us to uniquely identify our preference. These fields are reused when we read the
value of the preference. The
<code>string</code>
field tells the lua engine that the preference is a string. It could also be an integer,
a filename or any of the types detailed in API manual relating to
<code>types_lua_pref_type</code>
. We then have the label for the preference in the preference menu, the tooltip when
hovering over the value and a default value.
</para>
<para>
<code>darktable.register_storage</code>
is the call that actually registers a new storage. The first argument is a name for the
storage, the second is the label that will be displayed in the UI and last is a function
to call on each image. This function has a lot of parameters, but
<code>filename</code>
is the only one we use in this example. It contains the name of a temporary file where
the image was exported by darktable's engine.
</para>
<para>
This code will work but it has a couple of limitations. This is just a simple example
after all:
<itemizedlist>
<listitem><para>
We use preferences to configure the target path. It would be nicer to add an element
to the export UI in darktable. We will detail how to do that in the next section
</para></listitem>
<listitem><para>
We do not check the returned value of scp. That command might fail, in particular if
the user has not correctly set the preference.
</para></listitem>
<listitem><para>
This script can't read input from the user. The remote scp must use password-less
copy. Scp can't be provided a password easily, so we will leave it that way
</para></listitem>
<listitem><para>
There is no message displayed once the example is done, only the progressbar on the
lower left side tells the user that the job is done.
</para></listitem>
<listitem><para>
We use
<code>coroutine.yield</code>
to call an external program. The normal
<code>os.execute</code>
would block other lua codes from happening.
</para></listitem>
</itemizedlist>
</para>
</sect2>
<sect2 id="lua_gui_example">
<title>Building User Interface elements</title>
<para>
Our previous example was a bit limited. In particular the use of a preference for the
export path wasn't very nice. We can do better than that by adding elements to the user
interface in the export dialog.
</para>
<para>
UI elements are created via the
<code>darktable_new_widget</code>
function. This function takes a type of widget as a parameter and returns a new object
corresponding to that widget. You can then set various fields to that widget to set its
parameters. You will then use that object as a parameter to various functions that will
add it to the darktable UI. The following simple example adds a lib in the lighttable
view with a simple label
</para>
<programlisting language="lua">
local my_label = darktable.new_widget("label")
my_label.label = "Hello, world !"
dt.register_lib("test","test",false,{
[dt.gui.views.lighttable] = {"DT_UI_CONTAINER_PANEL_LEFT_CENTER",20},
},my_label)
</programlisting>
<para>
There is a nice syntactic trick to make UI element code easier to read and write. You
can call these objects as functions with a table of key values as an argument. This
allows the following example to work. It creates a container widget with two
sub-widgets. A label and a text entry field.
</para>
<programlisting language="lua">
local my_widget = darktable.new_widget("box"){
orientation = "horizontal",
darktable.new_widget("label"){ label = "here => " },
darktable.new_widget("entry"){ tooltip = "please enter text here" }
}
</programlisting>
<para>
Now that we know that, let's improve our script a bit.
</para>
<programlisting language="lua">
darktable = require "darktable"
local scp_path = darktable.new_widget("entry"){
tooltip ="Complete path to copy to. Can include user and hostname",
text = "",
reset_callback = function(self) self.text = "" end
}
darktable.register_storage("scp_export","Export via scp",
function( storage, image, format, filename,
number, total, high_quality, extra_data)
if coroutine.yield("RUN_COMMAND","scp "..filename.." "..
scp_path.text
) then
darktable.print_error("scp failed for "..tostring(image))
end
end,
nil, --finalize
nil, --supported
nil, --initialize
darktable.new_widget("box") {
orientation ="horizontal",
darktable.new_widget("label"){label = "target SCP PATH "},
scp_path,
})
</programlisting>
</sect2>
<sect2 id="lua_sharing_scripts">
<title>Sharing scripts</title>
<para>
So far, our lua code was in <emphasis>luarc</emphasis>. That's a good way to develop
your script but not very practical for distribution. We need to make this into a proper
lua module. To do that, we save the code in a separate file
(<command>scp-storage.lua</command> in our case):
</para>
<para>
<programlisting language="lua">
--[[
SCP STORAGE
a simple storage to export images via scp
AUTHOR
Jérémy Rosen (jeremy.rosen@enst-bretagne.fr)
INSTALLATION
* copy this file in $CONFIGDIR/lua/ where CONFIGDIR
is your darktable configuration directory
* add the following line in the file $CONFIGDIR/luarc
require "scp-storage"
USAGE
* select "Export via SCP" in the storage selection menu
* set the target directory
* export your images
LICENSE
GPLv2
]]
darktable = require "darktable"
darktable.configuration.check_version(...,{2,0,0})
local scp_path = darktable.new_widget("entry"){
tooltip ="Complete path to copy to. Can include user and hostname",
text = "",
reset_callback = function(self) self.text = "" end
}
darktable.register_storage("scp_export","Export via scp",
function( storage, image, format, filename,
number, total, high_quality, extra_data)
if coroutine.yield("RUN_COMMAND","scp "..filename.." "..
scp_path.text
) then
darktable.print_error("scp failed for "..tostring(image))
end
end,
nil, --finalize
nil, --supported
nil, --initialize
darktable.new_widget("box") {
orientation ="horizontal",
darktable.new_widget("label"){label = "target SCP PATH "},
scp_path,
})
</programlisting>
</para>
<para>
darktable will look for scripts (following the normal lua rules) in the standard
directories plus
<code>$CONFIGDIR/lua/?.lua</code>
. So our script can be called by simply adding
<code>require "scp-storage"</code>
in the <emphasis>luarc</emphasis> file. A couple of extra notes...
<itemizedlist>
<listitem><para>
The function
<code>darktable.configuration.check_version</code>
will check compatibility for you. The
<code>...</code>
will turn into your script's name and
<code>{2,0,0}</code>
is the API version you have tested your script with. You can add multiple API
versions if you update your script for multiple versions of darktable.
</para></listitem>
<listitem><para>
Make sure to declare all your functions as
<code>local</code>
to not pollute the general namespace.
</para></listitem>
<listitem><para>
Make sure you do not leave debug prints in your code.
<code>darktable.print_error</code>
in particular allows you to leave debug prints in your final code without disturbing
the console.
</para></listitem>
<listitem><para>
You are free to choose any license for your script but scripts that are uploaded on
darktable's website need to be GPLv2.
</para></listitem>
</itemizedlist>
Once you have filled all the fields, checked your code, you can upload it to our script
page
<ulink url="http://darktable.org/redmine/projects/darktable/wiki/LuaScripts">here</ulink>.
</para>
</sect2>
<sect2 id="lua_dbus">
<title>Calling Lua from DBus</title>
<para>
It is possible to send a lua command to darktable via its DBus interface. The method
<emphasis>org.darktable.service.Remote.Lua</emphasis> takes a single string parameter
which is interpreted as a lua command. The command will be executed in the current lua
context and should return either <emphasis>nil</emphasis> or a string. The result will
be passed back as the result of the DBus method.
</para>
<para>
If the Lua call results in an error, the DBus method call will return an error
<emphasis>org.darktable.Error.LuaError</emphasis> with the lua error message as the
message attached to the DBus error.
</para>
</sect2>
<sect2 id="lua_lib">
<title>Using darktable from a lua script</title>
<para>
<emphasis>Warning: This feature is very experimental. It is known that several elements
don't work in library mode yet. Careful testing is highly recommended.</emphasis>
</para>
<para>
The lua interface allows you to use darktable from any lua script. This will load
darktable as a library and provide you with most of the lua API (darktable is configured
headless, so the functions related to the user interface are not available).
</para>
<para>
As an example, the following program will print the list of all images in your library:
</para>
<para>
<programlisting language="lua">#!/usr/bin/env lua
package = require "package"
package.cpath=package.cpath..";./lib/darktable/lib?.so"
dt = require("darktable")(
"--library", "./library.db",
"--datadir", "./share/darktable",
"--moduledir", "./lib/darktable",
"--configdir", "./configdir",
"--cachedir","cachedir",
"--g-fatal-warnings")
require("darktable.debug")
for k,v in ipairs(dt.database) do
print(tostring(v))
end
</programlisting>
</para>
<para>
Note the third line that points to the location of the
<filename>libdarktable.so</filename> file.
</para>
<para>
Also note that the call to
<function>require</function>
returns a function that can be called only once and allows you to set darktable's
command line parameter. The
<function>:memory:</function>
parameter to
<function>--library</function>
is usefull here if you don't want to work on your personal library.
</para>
</sect2>
</sect1>
<sect1 status="final" id="lua_api">
<title>Lua API</title>
<indexterm>
<primary>Lua API</primary>
</indexterm>
<para>
darktable's Lua API is documented in its own manual with a detailed description of all
data structures and functions. You can download the API manual from
<ulink url="http://www.darktable.org/resources/">darktable's webpage</ulink>.
</para>
</sect1>
</chapter>
