Documentation
Data sources
Table of Contents ▼
Presentation
Temma offers a mechanism for managing access to data sources, making it easy to connect to them and use them in a unified way, while still allowing you to call on capabilities specific to each source.
Unified operation means that most data sources can be used in the same way, to write and read key/value pairs. Data sources used in this way can be interchanged (for example, a Memcache access replaced by MySQL or Redis; or a Beanstalkd message queue transparently replaced by AWS SQS).
Configuration
To open a connection to a data source, you need to configure it in the temma.json file (see configuration documentation).
{
// application configuration
"application": {
"dataSources": {
"db": "mysql://user:passwd@localhost/my_base",
"ndb": "redis://localhost",
"cache": "memcache://localhost",
"file": "file:///var/data/temma",
"s3": "s3://ACCESS_KEY:SECRET_KEY@REGION/my_bucket",
"sqs": "sqs://ACCESS_KEY:SECRET_KEY@my_queue",
"mq": "beanstalk://localhost/my_queue",
"sms": "smsmode://API_KEY",
"push": "pushover://APP_TOKEN",
"myDS1": "[MyDataSource]mydsn://SOME_PARAM",
"myDS2": "[\\Namespace\\OtherDataSource]otherdsn://OTHER_PARAM"
}
}
}
- Line 5: Connection to MySQL.
- Line 6: Connection to Redis.
- Line 7: Connection to Memcache.
- Line 8: Access to local files.
- Line 9: Connection to Amazon S3.
- Line 10: Connection to Amazon SQS.
- Line 11: Connection to Beanstalkd.
- Line 12: Connection to smsmode.
- Line 13: Connection to Pushover.
- Line 14: Use the MyDataSource object as data source.
- Line 15: Use the \Namespace\OtherDataSource object as data source.
See the documentation for each data source to learn more about its configuration.
In controllers, data sources are directly accessible using the names defined in the configuration: $this->db, $this->cache, $this->s3, etc.
Objects accessing the dependency injection component must pass through the dataSources array: $this->_loader->dataSources['db']
Unified access
Data sources offer a number of common methods for accessing key/value pairs. Some methods are not always usable (for example, Memcache, Amazon SQS or Beanstalkd don't allow you to search through stored data), but the principles are broadly the same.
There are three sets of methods:
- General methods (find out if data exists, delete data, etc.).
- Methods for reading and writing complex data. Data is serialized (usually in JSON format, unless the data source provides its own serialization mechanism).
- Methods for reading and writing raw data. These methods manipulate strings which are stored directly.
For simplicity's sake, complex data can be accessed using associative array syntax.
Table-type access
Existence of data
Use the isset() function.
if (isset($this->source['key1']))
TµLog::l("The 'key1' data exists.");
Reading data
The data is deserialized.
$data = $this->source['key1'];
Writing data
The data is serialized.
$this->source['key1'] = $data;
Deleting data
Use the unset() function.
unset($this->source['key1']);
Know the number of stored data
Use the count() function.
$nbr = count($this->source);
General methods
isSet()
isSet(string $key) : bool
Returns true if the key $key refers to data that exists.
Example:
// Check if the "key1" key exists
if (!$this->db->isSet('key1'))
TµLog::l("Unknown data.");
// alternate writing
if (!isset($this->db['key1']))
TµLog::l("Unknown data.");
remove()
remove(string $key) : void
Deletes the data referenced by the key $key.
Example:
// delete the key "key1"
$this->cache->remove('key1');
mRemove()
mRemove(array $keys) : void
Deletes all data whose keys are listed in the $keys array.
Example:
// delete the keys "key1" and "key2"
$this->cache->mRemove(['key1', 'key2']);
clear()
clear(string $pattern) : void
Deletes all data whose key corresponds to the parameter supplied. Depending on how the data source works, the parameter may be a prefix or a regular expression mask.
Example:
// delete all files in the "images" directory
$this->files->clear('images/*');
flush()
flush() : void
Deletes all data stored in the data source.
Example:
// delete all files stored in an Amazon S3 bucket
$this->s3->flush();
Management of complex serialized data
search()
search(string $pattern, bool $getValues=false) : array
Returns the list of keys corresponding to the first parameter. Depending on how the data source works, the parameter may be a prefix or a regular expression mask.
If the second parameter is set to true, the list returned contains the (deserialized) data values associated with each key.
Examples:
// retrieves the list of file names in the "users" directory
$users = $this->files->search('users/*');
foreach ($users as $login)
TµLog::l("User: $login");
// retrieves the list of files in the "users" directory,
// with their contents deserialized
$users = $this->files->search('users/*', true);
foreach ($users as $path => $user) {
$login = mb_substr($path, mb_strlen('users/'));
TµLog::l("User : $login");
TµLog::l("Age : " . $user['age']});
TµLog::l("Address: " . $user['address']);
}
get()
get(string $key, mixed $defaultOrCallback=null, mixed $options=null) : mixed
Returns the (deserialized) data whose key is supplied as the first parameter.
If the data does not exist in the data source, the second parameter is used:
- if it contains scalar data, this is used as the default value and returned by the method;
- if it contains a function, the function is executed. Its return value is added to the data source, associated with the supplied key, and returned by the method. In this case, the third parameter can receive options that are used to store the data.
Examples:
// retrieve cached data
$colors = $this->cache->get('app:colors');
// alternate writing
$color = $this->cache['app:colors'];
// idem, but if the data is not cached, a default value is returned
$colors = $this->cache->get('app:colors', ['blue', 'red', 'green']);
// $colors = ['blue', 'red', 'green'];
// Retrieve cached data. If the data is not cached, a function is called
// to fetch the data from the database (via a DAO). The cached data is
// saved and returned. The data will be available for subsequent accesses
// to the cache.
$user = $this->cache->get($userId, function() use ($userId) {
return $this->_dao->get($userId);
});
mGet()
mGet(array $keys) : array
Takes a list of keys as parameter, and returns an associative array with the keys and their (deserialized) contents.
Example:
// récupère les données (désérialisées) des clés "key1" et "key2"
$data = $this->db->mGet(['key1', 'key2']);
foreach ($data as $key => $datum) {
TµLog::l("Name: '$key' - size: '{$datum['size']}'.");
}
set()
set(string $key, mixed $value=null, mixed $options=null) : mixed
Writes (serialized) data to the data source.
- 1st parameter: Key of data to be created or updated.
- 2nd parameter: Data content (scalar or complex).
- 3rd parameter: Option used for recording, depending on the type of data source.
Example:
// add data to cache
$this->cache->set('app:color', 'blue');
// alternate writing
$this->cache['app:color'] = 'blue';
// create a text file on Amazon S3 with public read rights
$this->s3->set('text/introduction.txt', $texte, [
'public' => true,
'mimetype' => 'text/plain',
]);
mSet()
mSet(array $data, mixed $options=null) : int
Writes multiple (serialized) data.
- 1st parameter: Associative array whose keys are the identifiers of the data to be written, and whose associated values are the contents of the data to be written.
- 2nd parameter: Option used for all records, depending on data source type.
Example:
// write multiple (serialized) keys to a Redis database
$this->ndb->mSet([
'key1' => 'value 1',
'key2' => ['value 2.1', 'value 2.2', 'value 2.3'],
'key3' => [
'key3.1' => 'value 3.1',
'key3.2' => 'value 3.2',
],
]);
Raw data management
find()
find(string $pattern, bool $getValues=false) : array
Returns the list of keys corresponding to the first parameter. Depending on how the data source works, the parameter may be a prefix or a regular expression mask.
If the second parameter is set to true, the list returned contains the (raw) data values associated with each key.
If the second parameter is set to false, this method is identical to the search() method.
Examples:
// retrieves the list of file names in the "images" directory
$keys = $this->files->find('images/*');
foreach ($keys as $key)
TµLog::l("File: '$key'.");
// retrieves the list of files in the "images" directory,
// with their raw contents
$keys = $this->files->find('images/*', true);
foreach ($keys as $key => $value) {
$size = mb_strlen($value, 'ascii');
TµLog::l("File: '$key' - size: '$size'.");
}
read()
read(string $key, mixed $defaultOrCallback=null, mixed $options=null) : mixed
Returns the data whose key is supplied as the first parameter.
If the data does not exist in the data source, the second parameter is used:
- if it contains scalar data, this is used as the default value and returned by the method;
- if it contains a function, the function is executed. Its return value is added to the data source, associated with the supplied key, and returned by the method. In this case, the third parameter can receive options that are used to store the data.
Example:
// Retrieving a cached CSS file. If the file is not cached,
// it is read and added to the cache with a lifetime of 10 minutes.
$user = $this->cache->read('style.css', function() {
$css = file_get_contents('/path/to/style.css');
return ($css);
}, 600);
mRead()
mRead(array $keys) : array
Takes a list of keys as parameter, and returns an associative array with the keys and their contents.
Example:
// read multiple raw data from Redis
$data = $this->ndb->mRead(['key1', 'key2', 'key3']);
copyFrom()
copyFrom(string $key, string $localPath, mixed $defaultOrCallback=null, mixed $options=null) : bool
Retrieves data and saves it in a local file.
If the data doesn't exist, the third and fourth parameters are used to return a default value or to generate the data (see the read method).
Example:
// retrieves a file from Amazon S3 and saves it locally
$this->s3->copyFrom('images/user1.jpg', '/var/data/img/login.jpg');
mCopyFrom()
mCopyFrom(array $keys) : int
Takes as parameter an associative array whose keys are the identifiers of the data to be retrieved, and whose associated values are the paths of the files to which the data is to be written.
Example:
// retrieves multiple files from Amazon S3 and saves them locally
$this->s3->mCopyFrom([
'images/user1.jpg' => '/var/data/img/login1.jpg',
'images/user2.jpg' => '/var/data/img/login2.jpg',
]);
write()
write(string $key, string $value, mixed $options=null) : mixed
Writes data to the data source.
- 1st parameter: Key for data to be created or updated.
- 2nd parameter: Textual (or binary) data content.
- 3rd parameter: Possible option used for recording, depending on the type of data source.
Examples:
// writes data to a Redis server
$this->ndb->write('linux:year', '1991');
// writes data to Redis, with a lifetime of 10 minutes
$this->ndb->write('linux:year', '1991', 600);
mWrite()
mWrite(array $data, mixed $options=null) : int
Writes multiple data.
- 1st parameter: Associative array whose keys are the identifiers of the data to be written, and whose associated values are the contents of the data to be written.
- 2nd parameter: Option used for all records, depending on data source type.
Example:
// writes multiple data to a Redis server
$this->ndb->mWrite([
'qnx' => '1984',
'nextstep' => '1988',
'solaris' => '1990',
'linux' => '1991',
]);
copyTo()
copyTo(string $key, string $localPath, mixed $options=null) : mixed
Writes data to the data source, copying the contents of a local file.
- 1st parameter: Key to data to be created or updated.
- 2nd parameter: Path to source file.
- 3rd parameter: Possible option used for recording, depending on the type of data source.
Example:
// copy a local file to Amazon S3
$this->ndb->copyTo('css/style.css', '/var/data/css/style.css');
mCopyTo()
mCopyTo(array $data, mixed $options=null) : int
Writes multiple data sets, copying the contents of several local files.
- 1st parameter: Associative array whose keys are the identifiers of the data to be written, and whose associated values are the paths to the source files.
- 2nd parameter: Option used for all records, depending on data source type.
Example:
// copy multiple local files to Amazon S3
$this->ndb->mCopyTo([
'css/style.css' => '/var/data/css/style.css',
'js/app.js' => '/var/data/js/app.js',
'images/login.jpg' => '/var/data/img/login.jpg',
]);
Previous: | Sessions |
Next: | SQL |
Table of Contents
- Migration : How to upgrade from Temma 1.x to version 2
- Installation : Download Temma and install it to start your web project
- Configuration : All the configuration directives of the etc/temma.json file and the optional environment variables
- External libraries : How to use external function libraries
- Routing : Temma's default routing system, and advanced routing
- Log : Use of the log system, management by criticality levels
- Controllers : Essential parts of your web application
- Views : Smarty templates or JSON/CSV/RSS/iCal/INI exports
- Dependency injection : The backbone of your application developments
- Sessions : Extension of user sessions managed by PHP
-
Data sources :
Unified data access management
- SQL : Access to relational databases
- Memcache : Access to Memcached servers
- Redis : Access to Redis servers
- File : Access to local file storage
- S3 : Access to Amazon S3 storage
- Socket : Network communication
- SQS : Access to Amazon SQS message queues
- Beanstalk : Access to Beanstalkd message queue servers
- Smsmode : To send text messages to mobile phones
- Slack : To send notifications on Slack
- Pushover : To send push notifications to cell phones
- Model : How to use DAOs to access databases
- Execution flow : How to manage the execution flow, between plugins and the controller
- Plugins : How to use plugins, and create your own plugins to modularize your code
- Attributes : How to filter access to controllers and actions
- Tests : To write automated integration tests.
- Command-line interface : To create executable scripts on the command line, automatically initialized by Temma
- Helpers : Items offered by Temma to help you in various circumstances