No OneTemporary
Actions

Size

48 KB

Referenced Files

None

Subscribers

None

View Options

	diff --git a/src/docs/user/cluster/cluster_partitioning.diviner b/src/docs/user/cluster/cluster_partitioning.diviner
	index 1579d9d922..54aca0e82f 100644
	--- a/src/docs/user/cluster/cluster_partitioning.diviner
	+++ b/src/docs/user/cluster/cluster_partitioning.diviner
	@@ -1,204 +1,242 @@
	@title Cluster: Partitioning and Advanced Configuration
	@group cluster

	Guide to partitioning Phabricator applications across multiple database hosts.

	Overview
	========

	WARNING: Partitioning is a prototype.

	You can partition Phabricator's applications across multiple databases. For
	example, you can move an application like Files or Maniphest to a dedicated
	database host.

	The advantages of doing this are:

	- moving heavily used applications to dedicated hardware can help you
	scale; and
	- you can match application workloads to hardware or configuration to make
	operating the cluster easier.

	This configuration is complex, and very few installs need to pursue it.
	Phabricator will normally run comfortably with a single database master even
	for large organizations.

	Partitioning generally does not do much to increase resiliance or make it
	easier to recover from disasters, and is primarily a mechanism for scaling.

	If you are considering partitioning, you likely want to configure replication
	with a single master first. Even if you choose not to deploy replication, you
	should review and understand how replication works before you partition. For
	details, see @{Cluster:Databases}.

	+Databases also support some advanced configuration options. Briefly:
	+
	+ - `persistent`: Allows use of persistent connections, reducing pressure on
	+ outbound ports.
	+
	+See "Advanced Configuration", below, for additional discussion.
	+

	What Partitioning Does
	======================

	When you partition Phabricator, you move all of the data for one or more
	applications (like Maniphest) to a new master database host. This is possible
	because Phabricator stores data for each application in its own logical
	database (like `phabricator_maniphest`) and performs no joins between databases.

	If you're running into scale limits on a single master database, you can move
	one or more of your most commonly-used applications to a second database host
	and continue adding users. You can keep partitioning applications until all
	heavily used applications have dedicated database servers.

	Alternatively or additionally, you can partition applications to make operating
	the cluster easier. Some applications have unusual workloads or requirements,
	and moving them to separate hosts may make things easier to deal with overall.

	For example: if Files accounts for most of the data on your install, you might
	move it to a different host to make backing up everything else easier.


	Configuration Overview
	======================

	To configure partitioning, you will add multiple entries to `cluster.databases`
	with the `master` role. Each `master` should specify a new `partition` key,
	which contains a list of application databases it should host.

	One master may be specified as the `default` partition. Applications not
	explicitly configured to be assigned elsewhere will be assigned here.

	When you define multiple `master` databases, you must also specify which master
	each `replica` database follows. Here's a simple example config:

	```lang=json
	...
	"cluster.databases": [
	{
	"host": "db001.corporation.com",
	"role": "master",
	"user": "phabricator",
	"pass": "hunter2!trustno1",
	"port": 3306,
	"partition": [
	"default"
	]
	},
	{
	"host": "db002.corporation.com",
	"role": "replica",
	"user": "phabricator",
	"pass": "hunter2!trustno1",
	"port": 3306,
	"master": "db001.corporation.com:3306"
	},
	{
	"host": "db003.corporation.com",
	"role": "master",
	"user": "phabricator",
	"pass": "hunter2!trustno1",
	"port": 3306,
	"partition": [
	"file",
	"passphrase",
	"slowvote"
	]
	},
	{
	"host": "db004.corporation.com",
	"role": "replica",
	"user": "phabricator",
	"pass": "hunter2!trustno1",
	"port": 3306,
	"master": "db003.corporation.com:3306"
	}
	],
	...
	```

	In this configuration, `db001` is a master and `db002` replicates it.
	`db003` is a second master, replicated by `db004`.

	Applications have been partitioned like this:

	- `db003`/`db004`: Files, Passphrase, Slowvote
	- `db001`/`db002`: Default (all other applications)

	Not all of the database partition names are the same as the application
	names. You can get a list of databases with `bin/storage databases` to identify
	the correct database names.

	After you have configured partitioning, it needs to be committed to the
	databases. This writes a copy of the configuration to tables on the databases,
	preventing errors if a webserver accidentally starts with an old or invalid
	configuration.

	To commit the configuration, run this command:

	```
	phabricator/ $ ./bin/storage partition
	```

	Run this command after making any partition or clustering changes. Webservers
	will not serve traffic if their configuration and the database configuration
	differ.


	Launching a new Partition
	=========================

	To add a new partition, follow these steps:

	- Set up the new database host or hosts.
	- - Add the new database to `cluster.database`, but keep its "partition"
	+ - Add the new database to `cluster.databases`, but keep its "partition"
	configuration empty (just an empty list). If this is the first time you
	are partitioning, you will need to configure your existing master as the
	new "default". This will let Phabricator interact with it, but won't send
	any traffic to it yet.
	- Run `bin/storage partition`.
	- Run `bin/storage upgrade` to initialize the schemata on the new hosts.
	- Stop writes to the applications you want to move by putting Phabricator
	in read-only mode, or shutting down the webserver and daemons, or telling
	everyone not to touch anything.
	- Dump the data from the application databases on the old master.
	- Load the data into the application databases on the new master.
	- Reconfigure the "partition" setup so that Phabricator knows the databases
	have moved.
	- Run `bin/storage partition`.
	- While still in read-only mode, check that all the data appears to be
	intact.
	- Resume writes.

	You can do this with a small, rarely-used application first (on most installs,
	Slowvote might be a good candidate) if you want to run through the process
	end-to-end before performing a larger, higher-stakes migration.


	How Partitioning Works
	======================

	If you have multiple masters, Phabricator keeps the entire set of schemata up
	to date on all of them. When you run `bin/storage upgrade` or other storage
	management commands, they generally affect all masters (if they do not, they
	will prompt you to be more specific).

	When the application goes to read or write normal data (for example, to query a
	list of tasks) it only connects to the master which the application it is
	acting on behalf of is assigned to.

	In most cases, a masters will not have any data in most the databases which are
	not assigned to it. If they do (for example, because they previously hosted the
	application) the data is ignored. This approach (of maintaining all schemata on
	all hosts) makes it easier to move data and to quickly revert changes if a
	configuration mistake occurs.

	There are some exceptions to this rule. For example, all masters keep track
	of which patches have been applied to that particular master so that
	`bin/storage upgrade` can upgrade hosts correctly.

	Phabricator does not perform joins across logical databases, so there are no
	meaningful differences in runtime behavior if two applications are on the same
	physical host or different physical hosts.


	+Advanced Configuration
	+======================
	+
	+Separate from partitioning, some advanced configuration is supported. These
	+options must be set on database specifications in `cluster.databases`. You can
	+configure them without actually building a cluster by defining a cluster with
	+only one master.
	+
	+`persistent` //(bool)// Enables persistent connections. Defaults to off.
	+
	+With persitent connections enabled, Phabricator will keep a pool of database
	+connections open between web requests and reuse them when serving subsequent
	+requests.
	+
	+The primary benefit of using persistent connections is that it will greatly
	+reduce pressure on how quickly outbound TCP ports are opened and closed. After
	+a TCP port closes, it normally can't be used again for about 60 seconds, so
	+rapidly cycling ports can cause resource exuastion. If you're seeing failures
	+because requests are unable to bind to an outbound port, enabling this option
	+is likely to fix the issue. This option may also slightly increase performance.
	+
	+The cost of using persistent connections is that you may need to raise the
	+MySQL `max_connections` setting: although Phabricator will make far fewer
	+connections, the connections it does make will be longer-lived. Raising this
	+setting will increase MySQL memory requirements and may run into other limits,
	+like `open_files_limit`, which may also need to be raised.
	+
	+Persistent connections are enabled per-database. If you always want to use
	+them, set the flag on each configured database in `cluster.databases`.
	+
	+
	Next Steps
	==========

	Continue by:

	- returning to @{article:Clustering Introduction}.
	diff --git a/src/infrastructure/cluster/PhabricatorClusterDatabasesConfigOptionType.php b/src/infrastructure/cluster/PhabricatorClusterDatabasesConfigOptionType.php
	index 0963a3ca0c..48e65dc938 100644
	--- a/src/infrastructure/cluster/PhabricatorClusterDatabasesConfigOptionType.php
	+++ b/src/infrastructure/cluster/PhabricatorClusterDatabasesConfigOptionType.php
	@@ -1,92 +1,93 @@
	<?php

	final class PhabricatorClusterDatabasesConfigOptionType
	extends PhabricatorConfigJSONOptionType {

	public function validateOption(PhabricatorConfigOption $option, $value) {
	if (!is_array($value)) {
	throw new Exception(
	pht(
	'Database cluster configuration is not valid: value must be a '.
	'list of database hosts.'));
	}

	foreach ($value as $index => $spec) {
	if (!is_array($spec)) {
	throw new Exception(
	pht(
	'Database cluster configuration is not valid: each entry in the '.
	'list must be a dictionary describing a database host, but '.
	'the value with index "%s" is not a dictionary.',
	$index));
	}
	}

	$masters = array();
	$map = array();
	foreach ($value as $index => $spec) {
	try {
	PhutilTypeSpec::checkMap(
	$spec,
	array(
	'host' => 'string',
	'role' => 'string',
	'port' => 'optional int',
	'user' => 'optional string',
	'pass' => 'optional string',
	'disabled' => 'optional bool',
	'master' => 'optional string',
	'partition' => 'optional list<string>',
	+ 'persistent' => 'optional bool',
	));
	} catch (Exception $ex) {
	throw new Exception(
	pht(
	'Database cluster configuration has an invalid host '.
	'specification (at index "%s"): %s.',
	$index,
	$ex->getMessage()));
	}

	$role = $spec['role'];
	$host = $spec['host'];
	$port = idx($spec, 'port');

	switch ($role) {
	case 'master':
	case 'replica':
	break;
	default:
	throw new Exception(
	pht(
	'Database cluster configuration describes an invalid '.
	'host ("%s", at index "%s") with an unrecognized role ("%s"). '.
	'Valid roles are "%s" or "%s".',
	$spec['host'],
	$index,
	$spec['role'],
	'master',
	'replica'));
	}

	if ($role === 'master') {
	$masters[] = $host;
	}

	// We can't guarantee that you didn't just give the same host two
	// different names in DNS, but this check can catch silly copy/paste
	// mistakes.
	$key = "{$host}:{$port}";
	if (isset($map[$key])) {
	throw new Exception(
	pht(
	'Database cluster configuration is invalid: it describes the '.
	'same host ("%s") multiple times. Each host should appear only '.
	'once in the list.',
	$host));
	}
	$map[$key] = true;
	}

	}

	}
	diff --git a/src/infrastructure/cluster/PhabricatorDatabaseRef.php b/src/infrastructure/cluster/PhabricatorDatabaseRef.php
	index e59f8d55bc..a4f0259060 100644
	--- a/src/infrastructure/cluster/PhabricatorDatabaseRef.php
	+++ b/src/infrastructure/cluster/PhabricatorDatabaseRef.php
	@@ -1,688 +1,721 @@
	<?php

	final class PhabricatorDatabaseRef
	extends Phobject {

	const STATUS_OKAY = 'okay';
	const STATUS_FAIL = 'fail';
	const STATUS_AUTH = 'auth';
	const STATUS_REPLICATION_CLIENT = 'replication-client';

	const REPLICATION_OKAY = 'okay';
	const REPLICATION_MASTER_REPLICA = 'master-replica';
	const REPLICATION_REPLICA_NONE = 'replica-none';
	const REPLICATION_SLOW = 'replica-slow';
	const REPLICATION_NOT_REPLICATING = 'not-replicating';

	const KEY_REFS = 'cluster.db.refs';
	const KEY_INDIVIDUAL = 'cluster.db.individual';

	private $host;
	private $port;
	private $user;
	private $pass;
	private $disabled;
	private $isMaster;
	private $isIndividual;

	private $connectionLatency;
	private $connectionStatus;
	private $connectionMessage;

	private $replicaStatus;
	private $replicaMessage;
	private $replicaDelay;

	private $healthRecord;
	private $didFailToConnect;

	private $isDefaultPartition;
	private $applicationMap = array();
	private $masterRef;
	private $replicaRefs = array();
	+ private $usePersistentConnections;

	public function setHost($host) {
	$this->host = $host;
	return $this;
	}

	public function getHost() {
	return $this->host;
	}

	public function setPort($port) {
	$this->port = $port;
	return $this;
	}

	public function getPort() {
	return $this->port;
	}

	public function setUser($user) {
	$this->user = $user;
	return $this;
	}

	public function getUser() {
	return $this->user;
	}

	public function setPass(PhutilOpaqueEnvelope $pass) {
	$this->pass = $pass;
	return $this;
	}

	public function getPass() {
	return $this->pass;
	}

	public function setIsMaster($is_master) {
	$this->isMaster = $is_master;
	return $this;
	}

	public function getIsMaster() {
	return $this->isMaster;
	}

	public function setDisabled($disabled) {
	$this->disabled = $disabled;
	return $this;
	}

	public function getDisabled() {
	return $this->disabled;
	}

	public function setConnectionLatency($connection_latency) {
	$this->connectionLatency = $connection_latency;
	return $this;
	}

	public function getConnectionLatency() {
	return $this->connectionLatency;
	}

	public function setConnectionStatus($connection_status) {
	$this->connectionStatus = $connection_status;
	return $this;
	}

	public function getConnectionStatus() {
	if ($this->connectionStatus === null) {
	throw new PhutilInvalidStateException('queryAll');
	}

	return $this->connectionStatus;
	}

	public function setConnectionMessage($connection_message) {
	$this->connectionMessage = $connection_message;
	return $this;
	}

	public function getConnectionMessage() {
	return $this->connectionMessage;
	}

	public function setReplicaStatus($replica_status) {
	$this->replicaStatus = $replica_status;
	return $this;
	}

	public function getReplicaStatus() {
	return $this->replicaStatus;
	}

	public function setReplicaMessage($replica_message) {
	$this->replicaMessage = $replica_message;
	return $this;
	}

	public function getReplicaMessage() {
	return $this->replicaMessage;
	}

	public function setReplicaDelay($replica_delay) {
	$this->replicaDelay = $replica_delay;
	return $this;
	}

	public function getReplicaDelay() {
	return $this->replicaDelay;
	}

	public function setIsIndividual($is_individual) {
	$this->isIndividual = $is_individual;
	return $this;
	}

	public function getIsIndividual() {
	return $this->isIndividual;
	}

	public function setIsDefaultPartition($is_default_partition) {
	$this->isDefaultPartition = $is_default_partition;
	return $this;
	}

	public function getIsDefaultPartition() {
	return $this->isDefaultPartition;
	}

	+ public function setUsePersistentConnections($use_persistent_connections) {
	+ $this->usePersistentConnections = $use_persistent_connections;
	+ return $this;
	+ }
	+
	+ public function getUsePersistentConnections() {
	+ return $this->usePersistentConnections;
	+ }
	+
	public function setApplicationMap(array $application_map) {
	$this->applicationMap = $application_map;
	return $this;
	}

	public function getApplicationMap() {
	return $this->applicationMap;
	}

	public function getPartitionStateForCommit() {
	$state = PhabricatorEnv::getEnvConfig('cluster.databases');
	foreach ($state as $key => $value) {
	// Don't store passwords, since we don't care if they differ and
	// users may find it surprising.
	unset($state[$key]['pass']);
	}

	return phutil_json_encode($state);
	}

	public function setMasterRef(PhabricatorDatabaseRef $master_ref) {
	$this->masterRef = $master_ref;
	return $this;
	}

	public function getMasterRef() {
	return $this->masterRef;
	}

	public function addReplicaRef(PhabricatorDatabaseRef $replica_ref) {
	$this->replicaRefs[] = $replica_ref;
	return $this;
	}

	public function getReplicaRefs() {
	return $this->replicaRefs;
	}


	public function getRefKey() {
	$host = $this->getHost();

	$port = $this->getPort();
	if (strlen($port)) {
	return "{$host}:{$port}";
	}

	return $host;
	}

	public static function getConnectionStatusMap() {
	return array(
	self::STATUS_OKAY => array(
	'icon' => 'fa-exchange',
	'color' => 'green',
	'label' => pht('Okay'),
	),
	self::STATUS_FAIL => array(
	'icon' => 'fa-times',
	'color' => 'red',
	'label' => pht('Failed'),
	),
	self::STATUS_AUTH => array(
	'icon' => 'fa-key',
	'color' => 'red',
	'label' => pht('Invalid Credentials'),
	),
	self::STATUS_REPLICATION_CLIENT => array(
	'icon' => 'fa-eye-slash',
	'color' => 'yellow',
	'label' => pht('Missing Permission'),
	),
	);
	}

	public static function getReplicaStatusMap() {
	return array(
	self::REPLICATION_OKAY => array(
	'icon' => 'fa-download',
	'color' => 'green',
	'label' => pht('Okay'),
	),
	self::REPLICATION_MASTER_REPLICA => array(
	'icon' => 'fa-database',
	'color' => 'red',
	'label' => pht('Replicating Master'),
	),
	self::REPLICATION_REPLICA_NONE => array(
	'icon' => 'fa-download',
	'color' => 'red',
	'label' => pht('Not A Replica'),
	),
	self::REPLICATION_SLOW => array(
	'icon' => 'fa-hourglass',
	'color' => 'red',
	'label' => pht('Slow Replication'),
	),
	self::REPLICATION_NOT_REPLICATING => array(
	'icon' => 'fa-exclamation-triangle',
	'color' => 'red',
	'label' => pht('Not Replicating'),
	),
	);
	}

	public static function getClusterRefs() {
	$cache = PhabricatorCaches::getRequestCache();

	$refs = $cache->getKey(self::KEY_REFS);
	if (!$refs) {
	$refs = self::newRefs();
	$cache->setKey(self::KEY_REFS, $refs);
	}

	return $refs;
	}

	public static function getLiveIndividualRef() {
	$cache = PhabricatorCaches::getRequestCache();

	$ref = $cache->getKey(self::KEY_INDIVIDUAL);
	if (!$ref) {
	$ref = self::newIndividualRef();
	$cache->setKey(self::KEY_INDIVIDUAL, $ref);
	}

	return $ref;
	}

	public static function newRefs() {
	$default_port = PhabricatorEnv::getEnvConfig('mysql.port');
	$default_port = nonempty($default_port, 3306);

	$default_user = PhabricatorEnv::getEnvConfig('mysql.user');

	$default_pass = PhabricatorEnv::getEnvConfig('mysql.pass');
	$default_pass = new PhutilOpaqueEnvelope($default_pass);

	$config = PhabricatorEnv::getEnvConfig('cluster.databases');

	return id(new PhabricatorDatabaseRefParser())
	->setDefaultPort($default_port)
	->setDefaultUser($default_user)
	->setDefaultPass($default_pass)
	->newRefs($config);
	}

	public static function queryAll() {
	$refs = self::getActiveDatabaseRefs();
	return self::queryRefs($refs);
	}

	private static function queryRefs(array $refs) {
	foreach ($refs as $ref) {
	$conn = $ref->newManagementConnection();

	$t_start = microtime(true);
	$replica_status = false;
	try {
	$replica_status = queryfx_one($conn, 'SHOW SLAVE STATUS');
	$ref->setConnectionStatus(self::STATUS_OKAY);
	} catch (AphrontAccessDeniedQueryException $ex) {
	$ref->setConnectionStatus(self::STATUS_REPLICATION_CLIENT);
	$ref->setConnectionMessage(
	pht(
	'No permission to run "SHOW SLAVE STATUS". Grant this user '.
	'"REPLICATION CLIENT" permission to allow Phabricator to '.
	'monitor replica health.'));
	} catch (AphrontInvalidCredentialsQueryException $ex) {
	$ref->setConnectionStatus(self::STATUS_AUTH);
	$ref->setConnectionMessage($ex->getMessage());
	} catch (AphrontQueryException $ex) {
	$ref->setConnectionStatus(self::STATUS_FAIL);

	$class = get_class($ex);
	$message = $ex->getMessage();
	$ref->setConnectionMessage(
	pht(
	'%s: %s',
	get_class($ex),
	$ex->getMessage()));
	}
	$t_end = microtime(true);
	$ref->setConnectionLatency($t_end - $t_start);

	if ($replica_status !== false) {
	$is_replica = (bool)$replica_status;
	if ($ref->getIsMaster() && $is_replica) {
	$ref->setReplicaStatus(self::REPLICATION_MASTER_REPLICA);
	$ref->setReplicaMessage(
	pht(
	'This host has a "master" role, but is replicating data from '.
	'another host ("%s")!',
	idx($replica_status, 'Master_Host')));
	} else if (!$ref->getIsMaster() && !$is_replica) {
	$ref->setReplicaStatus(self::REPLICATION_REPLICA_NONE);
	$ref->setReplicaMessage(
	pht(
	'This host has a "replica" role, but is not replicating data '.
	'from a master (no output from "SHOW SLAVE STATUS").'));
	} else {
	$ref->setReplicaStatus(self::REPLICATION_OKAY);
	}

	if ($is_replica) {
	$latency = idx($replica_status, 'Seconds_Behind_Master');
	if (!strlen($latency)) {
	$ref->setReplicaStatus(self::REPLICATION_NOT_REPLICATING);
	} else {
	$latency = (int)$latency;
	$ref->setReplicaDelay($latency);
	if ($latency > 30) {
	$ref->setReplicaStatus(self::REPLICATION_SLOW);
	$ref->setReplicaMessage(
	pht(
	'This replica is lagging far behind the master. Data is at '.
	'risk!'));
	}
	}
	}
	}
	}

	return $refs;
	}

	public function newManagementConnection() {
	return $this->newConnection(
	array(
	'retries' => 0,
	'timeout' => 2,
	));
	}

	public function newApplicationConnection($database) {
	return $this->newConnection(
	array(
	'database' => $database,
	));
	}

	public function isSevered() {
	// If we only have an individual database, never sever our connection to
	// it, at least for now. It's possible that using the same severing rules
	// might eventually make sense to help alleviate load-related failures,
	// but we should wait for all the cluster stuff to stabilize first.
	if ($this->getIsIndividual()) {
	return false;
	}

	if ($this->didFailToConnect) {
	return true;
	}

	$record = $this->getHealthRecord();
	$is_healthy = $record->getIsHealthy();
	if (!$is_healthy) {
	return true;
	}

	return false;
	}

	public function isReachable(AphrontDatabaseConnection $connection) {
	$record = $this->getHealthRecord();
	$should_check = $record->getShouldCheck();

	if ($this->isSevered() && !$should_check) {
	return false;
	}

	try {
	$connection->openConnection();
	$reachable = true;
	} catch (AphrontSchemaQueryException $ex) {
	// We get one of these if the database we're trying to select does not
	// exist. In this case, just re-throw the exception. This is expected
	// during first-time setup, when databases like "config" will not exist
	// yet.
	throw $ex;
	} catch (Exception $ex) {
	$reachable = false;
	}

	if ($should_check) {
	$record->didHealthCheck($reachable);
	}

	if (!$reachable) {
	$this->didFailToConnect = true;
	}

	return $reachable;
	}

	public function checkHealth() {
	$health = $this->getHealthRecord();

	$should_check = $health->getShouldCheck();
	if ($should_check) {
	// This does an implicit health update.
	$connection = $this->newManagementConnection();
	$this->isReachable($connection);
	}

	return $this;
	}

	public function getHealthRecord() {
	if (!$this->healthRecord) {
	$this->healthRecord = new PhabricatorDatabaseHealthRecord($this);
	}
	return $this->healthRecord;
	}

	public static function getActiveDatabaseRefs() {
	$refs = array();

	foreach (self::getMasterDatabaseRefs() as $ref) {
	$refs[] = $ref;
	}

	foreach (self::getReplicaDatabaseRefs() as $ref) {
	$refs[] = $ref;
	}

	return $refs;
	}

	public static function getAllMasterDatabaseRefs() {
	$refs = self::getClusterRefs();

	if (!$refs) {
	return array(self::getLiveIndividualRef());
	}

	$masters = array();
	foreach ($refs as $ref) {
	if ($ref->getIsMaster()) {
	$masters[] = $ref;
	}
	}

	return $masters;
	}

	public static function getMasterDatabaseRefs() {
	$refs = self::getAllMasterDatabaseRefs();
	return self::getEnabledRefs($refs);
	}

	public function isApplicationHost($database) {
	return isset($this->applicationMap[$database]);
	}

	public function loadRawMySQLConfigValue($key) {
	$conn = $this->newManagementConnection();

	try {
	$value = queryfx_one($conn, 'SELECT @@%Q', $key);
	$value = $value['@@'.$key];
	} catch (AphrontQueryException $ex) {
	$value = null;
	}

	return $value;
	}

	public static function getMasterDatabaseRefForApplication($application) {
	$masters = self::getMasterDatabaseRefs();

	$application_master = null;
	$default_master = null;
	foreach ($masters as $master) {
	if ($master->isApplicationHost($application)) {
	$application_master = $master;
	break;
	}
	if ($master->getIsDefaultPartition()) {
	$default_master = $master;
	}
	}

	if ($application_master) {
	$masters = array($application_master);
	} else if ($default_master) {
	$masters = array($default_master);
	} else {
	$masters = array();
	}

	$masters = self::getEnabledRefs($masters);
	$master = head($masters);

	return $master;
	}

	public static function newIndividualRef() {
	$default_user = PhabricatorEnv::getEnvConfig('mysql.user');
	$default_pass = new PhutilOpaqueEnvelope(
	PhabricatorEnv::getEnvConfig('mysql.pass'));
	$default_host = PhabricatorEnv::getEnvConfig('mysql.host');
	$default_port = PhabricatorEnv::getEnvConfig('mysql.port');

	return id(new self())
	->setUser($default_user)
	->setPass($default_pass)
	->setHost($default_host)
	->setPort($default_port)
	->setIsIndividual(true)
	->setIsMaster(true)
	- ->setIsDefaultPartition(true);
	+ ->setIsDefaultPartition(true)
	+ ->setUsePersistentConnections(false);
	}

	public static function getAllReplicaDatabaseRefs() {
	$refs = self::getClusterRefs();

	if (!$refs) {
	return array();
	}

	$replicas = array();
	foreach ($refs as $ref) {
	if ($ref->getIsMaster()) {
	continue;
	}

	$replicas[] = $ref;
	}

	return $replicas;
	}

	public static function getReplicaDatabaseRefs() {
	$refs = self::getAllReplicaDatabaseRefs();
	return self::getEnabledRefs($refs);
	}

	private static function getEnabledRefs(array $refs) {
	foreach ($refs as $key => $ref) {
	if ($ref->getDisabled()) {
	unset($refs[$key]);
	}
	}
	return $refs;
	}

	public static function getReplicaDatabaseRefForApplication($application) {
	$replicas = self::getReplicaDatabaseRefs();

	$application_replicas = array();
	$default_replicas = array();
	foreach ($replicas as $replica) {
	$master = $replica->getMaster();

	if ($master->isApplicationHost($application)) {
	$application_replicas[] = $replica;
	}

	if ($master->getIsDefaultPartition()) {
	$default_replicas[] = $replica;
	}
	}

	if ($application_replicas) {
	$replicas = $application_replicas;
	} else {
	$replicas = $default_replicas;
	}

	$replicas = self::getEnabledRefs($replicas);

	// TODO: We may have multiple replicas to choose from, and could make
	// more of an effort to pick the "best" one here instead of always
	// picking the first one. Once we've picked one, we should try to use
	// the same replica for the rest of the request, though.

	return head($replicas);
	}

	private function newConnection(array $options) {
	// If we believe the database is unhealthy, don't spend as much time
	// trying to connect to it, since it's likely to continue to fail and
	// hammering it can only make the problem worse.
	$record = $this->getHealthRecord();
	if ($record->getIsHealthy()) {
	$default_retries = 3;
	$default_timeout = 10;
	} else {
	$default_retries = 0;
	$default_timeout = 2;
	}

	$spec = $options + array(
	'user' => $this->getUser(),
	'pass' => $this->getPass(),
	'host' => $this->getHost(),
	'port' => $this->getPort(),
	'database' => null,
	'retries' => $default_retries,
	'timeout' => $default_timeout,
	+ 'persistent' => $this->getUsePersistentConnections(),
	);

	- return self::newRawConnection($spec);
	+ $is_cli = (php_sapi_name() == 'cli');
	+
	+ $use_persistent = false;
	+ if (!empty($spec['persistent']) && !$is_cli) {
	+ $use_persistent = true;
	+ }
	+ unset($spec['persistent']);
	+
	+ $connection = self::newRawConnection($spec);
	+
	+ // If configured, use persistent connections. See T11672 for details.
	+ if ($use_persistent) {
	+ $connection->setPersistent($use_persistent);
	+ }
	+
	+ // Unless this is a script running from the CLI, prevent any query from
	+ // running for more than 30 seconds. See T10849 for details.
	+ if (!$is_cli) {
	+ $connection->setQueryTimeout(30);
	+ }
	+
	+ return $connection;
	}

	public static function newRawConnection(array $options) {
	if (extension_loaded('mysqli')) {
	return new AphrontMySQLiDatabaseConnection($options);
	} else {
	return new AphrontMySQLDatabaseConnection($options);
	}
	}

	}
	diff --git a/src/infrastructure/cluster/PhabricatorDatabaseRefParser.php b/src/infrastructure/cluster/PhabricatorDatabaseRefParser.php
	index bc0ab99663..989095c52c 100644
	--- a/src/infrastructure/cluster/PhabricatorDatabaseRefParser.php
	+++ b/src/infrastructure/cluster/PhabricatorDatabaseRefParser.php
	@@ -1,216 +1,219 @@
	<?php

	final class PhabricatorDatabaseRefParser
	extends Phobject {

	private $defaultPort = 3306;
	private $defaultUser;
	private $defaultPass;

	public function setDefaultPort($default_port) {
	$this->defaultPort = $default_port;
	return $this;
	}

	public function getDefaultPort() {
	return $this->defaultPort;
	}

	public function setDefaultUser($default_user) {
	$this->defaultUser = $default_user;
	return $this;
	}

	public function getDefaultUser() {
	return $this->defaultUser;
	}

	public function setDefaultPass($default_pass) {
	$this->defaultPass = $default_pass;
	return $this;
	}

	public function getDefaultPass() {
	return $this->defaultPass;
	}

	public function newRefs(array $config) {
	$default_port = $this->getDefaultPort();
	$default_user = $this->getDefaultUser();
	$default_pass = $this->getDefaultPass();

	$refs = array();

	$master_count = 0;
	foreach ($config as $key => $server) {
	$host = $server['host'];
	$port = idx($server, 'port', $default_port);
	$user = idx($server, 'user', $default_user);
	$disabled = idx($server, 'disabled', false);

	$pass = idx($server, 'pass');
	if ($pass) {
	$pass = new PhutilOpaqueEnvelope($pass);
	} else {
	$pass = clone $default_pass;
	}

	$role = $server['role'];
	$is_master = ($role == 'master');

	+ $use_persistent = (bool)idx($server, 'persistent', false);
	+
	$ref = id(new PhabricatorDatabaseRef())
	->setHost($host)
	->setPort($port)
	->setUser($user)
	->setPass($pass)
	->setDisabled($disabled)
	- ->setIsMaster($is_master);
	+ ->setIsMaster($is_master)
	+ ->setUsePersistentConnections($use_persistent);

	if ($is_master) {
	$master_count++;
	}

	$refs[$key] = $ref;
	}

	$is_partitioned = ($master_count > 1);
	if ($is_partitioned) {
	$default_ref = null;
	$partition_map = array();
	foreach ($refs as $key => $ref) {
	if (!$ref->getIsMaster()) {
	continue;
	}

	$server = $config[$key];
	$partition = idx($server, 'partition');
	if (!is_array($partition)) {
	throw new Exception(
	pht(
	'Phabricator is configured with multiple master databases, '.
	'but master "%s" is missing a "partition" configuration key to '.
	'define application partitioning.',
	$ref->getRefKey()));
	}

	$application_map = array();
	foreach ($partition as $application) {
	if ($application === 'default') {
	if ($default_ref) {
	throw new Exception(
	pht(
	'Multiple masters (databases "%s" and "%s") specify that '.
	'they are the "default" partition. Only one master may be '.
	'the default.',
	$ref->getRefKey(),
	$default_ref->getRefKey()));
	} else {
	$default_ref = $ref;
	$ref->setIsDefaultPartition(true);
	}
	} else if (isset($partition_map[$application])) {
	throw new Exception(
	pht(
	'Multiple masters (databases "%s" and "%s") specify that '.
	'they are the partition for application "%s". Each '.
	'application may be allocated to only one partition.',
	$partition_map[$application]->getRefKey(),
	$ref->getRefKey(),
	$application));
	} else {
	// TODO: We should check that the application is valid, to
	// prevent typos in application names. However, we do not
	// currently have an efficient way to enumerate all of the valid
	// application database names.

	$partition_map[$application] = $ref;
	$application_map[$application] = $application;
	}
	}

	$ref->setApplicationMap($application_map);
	}
	} else {
	// If we only have one master, make it the default.
	foreach ($refs as $ref) {
	if ($ref->getIsMaster()) {
	$ref->setIsDefaultPartition(true);
	}
	}
	}

	$ref_map = array();
	$master_keys = array();
	foreach ($refs as $ref) {
	$ref_key = $ref->getRefKey();
	if (isset($ref_map[$ref_key])) {
	throw new Exception(
	pht(
	'Multiple configured databases have the same internal '.
	'key, "%s". You may have listed a database multiple times.',
	$ref_key));
	} else {
	$ref_map[$ref_key] = $ref;
	if ($ref->getIsMaster()) {
	$master_keys[] = $ref_key;
	}
	}
	}

	foreach ($refs as $key => $ref) {
	if ($ref->getIsMaster()) {
	continue;
	}

	$server = $config[$key];

	$partition = idx($server, 'partition');
	if ($partition !== null) {
	throw new Exception(
	pht(
	'Database "%s" is configured as a replica, but specifies a '.
	'"partition". Only master databases may have a partition '.
	'configuration. Replicas use the same configuration as the '.
	'master they follow.',
	$ref->getRefKey()));
	}

	$master_key = idx($server, 'master');
	if ($master_key === null) {
	if ($is_partitioned) {
	throw new Exception(
	pht(
	'Database "%s" is configured as a replica, but does not '.
	'specify which "master" it follows in configuration. Valid '.
	'masters are: %s.',
	$ref->getRefKey(),
	implode(', ', $master_keys)));
	} else if ($master_keys) {
	$master_key = head($master_keys);
	} else {
	throw new Exception(
	pht(
	'Database "%s" is configured as a replica, but there is no '.
	'master configured.',
	$ref->getRefKey()));
	}
	}

	if (!isset($ref_map[$master_key])) {
	throw new Exception(
	pht(
	'Database "%s" is configured as a replica and specifies a '.
	'master ("%s"), but that master is not a valid master. Valid '.
	'masters are: %s.',
	implode(', ', $master_keys)));
	}

	$master_ref = $ref_map[$master_key];
	$ref->setMasterRef($ref_map[$master_key]);
	$master_ref->addReplicaRef($ref);
	}

	return array_values($refs);
	}

	}
	diff --git a/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php b/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php
	index 240567ce28..743ca9b759 100644
	--- a/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php
	+++ b/src/infrastructure/storage/lisk/PhabricatorLiskDAO.php
	@@ -1,334 +1,319 @@
	<?php

	/**
	* @task config Configuring Storage
	*/
	abstract class PhabricatorLiskDAO extends LiskDAO {

	private static $namespaceStack = array();

	const ATTACHABLE = '<attachable>';
	const CONFIG_APPLICATION_SERIALIZERS = 'phabricator/serializers';

	/* -( Configuring Storage )------------------------------------------------ */

	/**
	* @task config
	*/
	public static function pushStorageNamespace($namespace) {
	self::$namespaceStack[] = $namespace;
	}

	/**
	* @task config
	*/
	public static function popStorageNamespace() {
	array_pop(self::$namespaceStack);
	}

	/**
	* @task config
	*/
	public static function getDefaultStorageNamespace() {
	return PhabricatorEnv::getEnvConfig('storage.default-namespace');
	}

	/**
	* @task config
	*/
	public static function getStorageNamespace() {
	$namespace = end(self::$namespaceStack);
	if (!strlen($namespace)) {
	$namespace = self::getDefaultStorageNamespace();
	}
	if (!strlen($namespace)) {
	throw new Exception(pht('No storage namespace configured!'));
	}
	return $namespace;
	}

	/**
	* @task config
	*/
	protected function establishLiveConnection($mode) {
	$namespace = self::getStorageNamespace();
	$database = $namespace.'_'.$this->getApplicationName();

	$is_readonly = PhabricatorEnv::isReadOnly();

	if ($is_readonly && ($mode != 'r')) {
	$this->raiseImproperWrite($database);
	}

	$connection = $this->newClusterConnection(
	$this->getApplicationName(),
	$database,
	$mode);

	// TODO: This should be testing if the mode is "r", but that would probably
	// break a lot of things. Perform a more narrow test for readonly mode
	// until we have greater certainty that this works correctly most of the
	// time.
	if ($is_readonly) {
	$connection->setReadOnly(true);
	}

	- // Unless this is a script running from the CLI:
	- // - (T10849) Prevent any query from running for more than 30 seconds.
	- // - (T11672) Use persistent connections.
	- if (php_sapi_name() != 'cli') {
	-
	- // TODO: For now, disable this until after T11044: it's better at high
	- // load, but causes us to use slightly more connections at low load and
	- // is pushing users over limits like MySQL "max_connections".
	- $use_persistent = false;
	-
	- $connection
	- ->setQueryTimeout(30)
	- ->setPersistent($use_persistent);
	- }
	-
	return $connection;
	}

	private function newClusterConnection($application, $database, $mode) {
	$master = PhabricatorDatabaseRef::getMasterDatabaseRefForApplication(
	$application);

	if ($master && !$master->isSevered()) {
	$connection = $master->newApplicationConnection($database);
	if ($master->isReachable($connection)) {
	return $connection;
	} else {
	if ($mode == 'w') {
	$this->raiseImpossibleWrite($database);
	}
	PhabricatorEnv::setReadOnly(
	true,
	PhabricatorEnv::READONLY_UNREACHABLE);
	}
	}

	$replica = PhabricatorDatabaseRef::getReplicaDatabaseRefForApplication(
	$application);
	if ($replica) {
	$connection = $replica->newApplicationConnection($database);
	$connection->setReadOnly(true);
	if ($replica->isReachable($connection)) {
	return $connection;
	}
	}

	if (!$master && !$replica) {
	$this->raiseUnconfigured($database);
	}

	$this->raiseUnreachable($database);
	}

	private function raiseImproperWrite($database) {
	throw new PhabricatorClusterImproperWriteException(
	pht(
	'Unable to establish a write-mode connection (to application '.
	'database "%s") because Phabricator is in read-only mode. Whatever '.
	'you are trying to do does not function correctly in read-only mode.',
	$database));
	}

	private function raiseImpossibleWrite($database) {
	throw new PhabricatorClusterImpossibleWriteException(
	pht(
	'Unable to connect to master database ("%s"). This is a severe '.
	'failure; your request did not complete.',
	$database));
	}

	private function raiseUnconfigured($database) {
	throw new Exception(
	pht(
	'Unable to establish a connection to any database host '.
	'(while trying "%s"). No masters or replicas are configured.',
	$database));
	}

	private function raiseUnreachable($database) {
	throw new PhabricatorClusterStrandedException(
	pht(
	'Unable to establish a connection to any database host '.
	'(while trying "%s"). All masters and replicas are completely '.
	'unreachable.',
	$database));
	}


	/**
	* @task config
	*/
	public function getTableName() {
	$str = 'phabricator';
	$len = strlen($str);

	$class = strtolower(get_class($this));
	if (!strncmp($class, $str, $len)) {
	$class = substr($class, $len);
	}
	$app = $this->getApplicationName();
	if (!strncmp($class, $app, strlen($app))) {
	$class = substr($class, strlen($app));
	}

	if (strlen($class)) {
	return $app.'_'.$class;
	} else {
	return $app;
	}
	}

	/**
	* @task config
	*/
	abstract public function getApplicationName();

	protected function getConnectionNamespace() {
	return self::getStorageNamespace().'_'.$this->getApplicationName();
	}


	/**
	* Break a list of escaped SQL statement fragments (e.g., VALUES lists for
	* INSERT, previously built with @{function:qsprintf}) into chunks which will
	* fit under the MySQL 'max_allowed_packet' limit.
	*
	* Chunks are glued together with `$glue`, by default ", ".
	*
	* If a statement is too large to fit within the limit, it is broken into
	* its own chunk (but might fail when the query executes).
	*/
	public static function chunkSQL(
	array $fragments,
	$glue = ', ',
	$limit = null) {

	if ($limit === null) {
	// NOTE: Hard-code this at 1MB for now, minus a 10% safety buffer.
	// Eventually we could query MySQL or let the user configure it.
	$limit = (int)((1024 * 1024) * 0.90);
	}

	$result = array();

	$chunk = array();
	$len = 0;
	$glue_len = strlen($glue);
	foreach ($fragments as $fragment) {
	$this_len = strlen($fragment);

	if ($chunk) {
	// Chunks after the first also imply glue.
	$this_len += $glue_len;
	}

	if ($len + $this_len <= $limit) {
	$len += $this_len;
	$chunk[] = $fragment;
	} else {
	if ($chunk) {
	$result[] = $chunk;
	}
	$len = strlen($fragment);
	$chunk = array($fragment);
	}
	}

	if ($chunk) {
	$result[] = $chunk;
	}

	foreach ($result as $key => $fragment_list) {
	$result[$key] = implode($glue, $fragment_list);
	}

	return $result;
	}

	protected function assertAttached($property) {
	if ($property === self::ATTACHABLE) {
	throw new PhabricatorDataNotAttachedException($this);
	}
	return $property;
	}

	protected function assertAttachedKey($value, $key) {
	$this->assertAttached($value);
	if (!array_key_exists($key, $value)) {
	throw new PhabricatorDataNotAttachedException($this);
	}
	return $value[$key];
	}

	protected function detectEncodingForStorage($string) {
	return phutil_is_utf8($string) ? 'utf8' : null;
	}

	protected function getUTF8StringFromStorage($string, $encoding) {
	if ($encoding == 'utf8') {
	return $string;
	}

	if (function_exists('mb_detect_encoding')) {
	if (strlen($encoding)) {
	$try_encodings = array(
	$encoding,
	);
	} else {
	// TODO: This is pretty much a guess, and probably needs to be
	// configurable in the long run.
	$try_encodings = array(
	'JIS',
	'EUC-JP',
	'SJIS',
	'ISO-8859-1',
	);
	}

	$guess = mb_detect_encoding($string, $try_encodings);
	if ($guess) {
	return mb_convert_encoding($string, 'UTF-8', $guess);
	}
	}

	return phutil_utf8ize($string);
	}

	protected function willReadData(array &$data) {
	parent::willReadData($data);

	static $custom;
	if ($custom === null) {
	$custom = $this->getConfigOption(self::CONFIG_APPLICATION_SERIALIZERS);
	}

	if ($custom) {
	foreach ($custom as $key => $serializer) {
	$data[$key] = $serializer->willReadValue($data[$key]);
	}
	}
	}

	protected function willWriteData(array &$data) {
	static $custom;
	if ($custom === null) {
	$custom = $this->getConfigOption(self::CONFIG_APPLICATION_SERIALIZERS);
	}

	if ($custom) {
	foreach ($custom as $key => $serializer) {
	$data[$key] = $serializer->willWriteValue($data[$key]);
	}
	}

	parent::willWriteData($data);
	}


	}

File Metadata

Mime Type: text/x-diff
Expires: Tue, Jul 29, 8:58 AM (3 w, 1 d ago)
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 188400
Default Alt Text: (48 KB)

No OneTemporaryActions

View Options

File Metadata

Event Timeline

No OneTemporary
Actions