Watchdog is a handy Python package which uses the inotify Linux kernel subsystem to watch for any changes to the filesystem. This makes it an excellent foundation to build a a small script which takes action whenever a file is received in a directory, or any of the directory's contents change. An example might be a client-facing sftp server where you may want to receive an email when a file is received.
You can install the package with below command:
First create the monitoring script, it will run daemonized and will observe any changes to the given directory. In that script 3 modules/classes will be used:
Each one of those methods receives the event object as first parameter, and the event object has 3 attributes:
So to create a handler just inherit from one of the existing handlers, for this example PatternMatchingEventHandler will be used to match only xml files. To simplify I will enclose the file processor in just one method, and I will implement method only for on_modified and on_created, which means that my handler will ignore any other events.
Also defining the patterns attribute to watch only for files with xml or lxml extensions.
That's all needed to watch for modifications on the given directory, it will take the current directory as default or the path given as first parameter.
Let it run in a shell and open another one or the file browser to change or create new .xml files in the /path/to/directory.
Since the handler is printing the results, the outrput should be:
Now to complete the script only need to implement in the process method, the necessary logic to parse and insert to database. For example, if the xml file contains some data about current track on a web radio:
The easiest way to parse this small xml is using xmltodict library.
With xmltodict.parse function the above xml will be outputed as an OrderedDict:
Much better than XPATH, and for this particular case when the xml_source is small there will no relevant performace issue. Now only need to get the values and populate the database, in my case I will use Redis DataModel as storage. Also I will use magicdate module to automagically convert the date format to datetime object. The complete code is as below:
* Using Python's Watchdog to monitor changes to a directory