Page MenuHomestyx hydra
Files F570660

No OneTemporary
Actions

View Options

diff --git a/scripts/symbols/generate_ctags_symbols.php b/scripts/symbols/generate_ctags_symbols.php
new file mode 100755
index 0000000000..4133c13fc8
--- /dev/null
+++ b/scripts/symbols/generate_ctags_symbols.php
@@ -0,0 +1,132 @@
+#!/usr/bin/env php
+<?php
+
+/*
+ * Copyright 2012 Facebook, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+$root = dirname(dirname(dirname(__FILE__)));
+require_once $root.'/scripts/__init_script__.php';
+
+phutil_require_module('phutil', 'console');
+
+if (ctags_check_executable() == false) {
+ echo phutil_console_format(
+ "Could not find Exuberant ctags. Make sure it is installed and\n".
+ "available in executable path.\n\n".
+ "Exuberant ctags project page: http://ctags.sourceforge.net/\n");
+ exit(1);
+}
+
+if ($argc !== 1 || posix_isatty(STDIN)) {
+ echo phutil_console_format(
+ "usage: find . -type f -name '*.py' | ./generate_ctags_symbols.php\n");
+ exit(1);
+}
+
+$input = file_get_contents('php://stdin');
+$input = trim($input);
+$input = explode("\n", $input);
+
+$data = array();
+$futures = array();
+
+foreach ($input as $file) {
+ $file = Filesystem::readablePath($file);
+ $futures[$file] = ctags_get_parser_future($file);
+}
+
+foreach (Futures($futures)->limit(8) as $file => $future) {
+ $tags = $future->resolve();
+ $tags = explode("\n", $tags[1]);
+
+ foreach ($tags as $tag) {
+ $parts = explode(";", $tag);
+ // skip lines that we can not parse
+ if (count($parts) < 2) {
+ continue;
+ }
+
+ // split ctags information
+ $tag_info = explode("\t", $parts[0]);
+ // split exuberant ctags "extension fields" (additional information)
+ $parts[1] = trim($parts[1], "\t \"");
+ $extension_fields = explode("\t", $parts[1]);
+
+ // skip lines that we can not parse
+ if (count($tag_info) < 3 || count($extension_fields) < 2) {
+ continue;
+ }
+
+ list($token, $file_path, $line_num) = $tag_info;
+ list($type, $language) = $extension_fields;
+
+ // strip "language:"
+ $language = substr($language, 9);
+
+ // To keep consistent with "Separate with commas, for example: php, py"
+ // in Arcanist Project edit form.
+ $language = str_ireplace("python", "py", $language);
+
+ // also, "normalize" c++ and c#
+ $language = str_ireplace("c++", "cpp", $language);
+ $language = str_ireplace("c#", "csharp", $language);
+
+ switch ($type) {
+ case 'class':
+ print_symbol($file_path, $line_num, 'class', $token, $language);
+ break;
+ case 'function':
+ print_symbol($file_path, $line_num, 'function', $token, $language);
+ break;
+ default:
+ }
+ }
+}
+
+function ctags_get_parser_future($file_path) {
+ $future = new ExecFuture('ctags -n --fields=Kl -o - %s',
+ $file_path);
+ return $future;
+}
+
+function ctags_check_executable() {
+ $future = new ExecFuture('ctags --version');
+ $result = $future->resolve();
+
+ if (empty($result[1])) {
+ return false;
+ }
+
+ return true;
+}
+
+function print_symbol($file, $line_num, $type, $token, $language) {
+ // get rid of relative path
+ $file = explode('/', $file);
+ if ($file[0] == '.' || $file[0] == "..") {
+ array_shift($file);
+ }
+ $file = '/' . implode('/', $file);
+
+ $parts = array(
+ $token,
+ $type,
+ strtolower($language),
+ $line_num,
+ $file,
+ );
+ echo implode(' ', $parts)."\n";
+}
diff --git a/src/docs/userguide/diffusion_symbols.diviner b/src/docs/userguide/diffusion_symbols.diviner
index d17f95560d..f0f0131960 100644
--- a/src/docs/userguide/diffusion_symbols.diviner
+++ b/src/docs/userguide/diffusion_symbols.diviner
@@ -1,84 +1,90 @@
@title Diffusion User Guide: Symbol Indexes
@group userguide
Guide to configuring and using the symbol index.
= Overview =
Phabricator can maintain a symbol index, which keeps track of where classes
and functions are defined in the codebase. Once you set up indexing, you can
use the index to do things like:
- link symbol uses in Differential code reviews to their definitions
- allow you to search for symbols
- let the IRC bot answer questions like "Where is SomeClass?"
NOTE: Symbol indexing is somewhat new, and has broader support for PHP than for
other languages.
= Populating the Index =
To populate the index, you need to write a script which identifies symbols in
your codebase and set up a cronjob which pipes its output to:
./scripts/symbols/import_project_symbols.php
Phabricator includes a script which can identify symbols in PHP projects:
./scripts/symbols/generate_php_symbols.php
+Phabricator also includes a script which can identify symbols in any
+programming language that has classes and/or functions, and is supported by
+Exuberant Ctags (http://ctags.sourceforge.net):
+
+ ./scripts/symbols/generate_ctags_symbols.php
+
If you want to identify symbols from another language, you need to write a
script which can export them (for example, maybe by parsing a ##ctags## file).
The output format of the script should be one symbol per line:
<name> <type> <lang> <line> <path>
For example:
ExampleClass class php 13 /src/classes/ExampleClass.php
Your script should enumerate all the symbols in your project, and provide paths
from the project root (where ".arcconfig" is) beginning with a "/". If there are
any duplicate symbols, it should include logic to pick the "best" one -- symbol
names must be unique within a project, type and language.
You can look at ##generate_php_symbols.php## for an example of how you might
write such a script, and run this command to see its output:
$ cd phabricator/
$ find . -type f -name '*.php' | ./scripts/symbols/generate_php_symbols.php
To actually build the symbol index, pipe this data to the
##import_project_symbols.php## script, providing the project name:
$ ./scripts/symbols/import_project_symbols.php yourproject < symbols_data
Then just set up a cronjob to run that however often you like.
You can test that the import worked by querying for symbols using the Conduit
method ##differential.findsymbols##. Some features (like that method, and the
IRC bot integration) will start working immediately. Others will require more
configuration.
= Configuring Differential Integration =
To configure Differential integration, you need to tell Phabricator which
projects have symbol indexes you want to use, and which other projects they
should pull symbols from. To do this, go to
##Repositories -> Arcanist Projects -> Edit## as an administrator. You need to
fill out these fields:
- **Repository**: Associate the project with a tracked repository.
- **Indexed Languages**: Fill in all the languages you've built indexes for.
- **Uses Symbols From**: If this project depends on other projects, add the
other projects which symbols should be looked for here. For example,
Phabricator lists "Arcanist" and "libphutil" because it uses classes and
functions from these projects.
Once you've configured a project, new revisions in that project will
automatically link symbols in Differential.
NOTE: Because this feature depends on the syntax highlighter, it will work
better for some languages than others. It currently works fairly well for PHP,
but your milage may vary for other languages.

File Metadata

Mime Type
text/x-diff
Expires
Sat, Sep 20, 2:16 AM (54 m, 6 s)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
240688
Default Alt Text
(7 KB)

Event Timeline

styx copyright 2020 sirocyl / ellie (kymkdd) / ncommander / styx contributors
Use of other names and brands is not intended to reflect a claim of, or right to, use the name or brand.
styx is an operating system using the Linux kernel and components, and "styx Linux" as a phrase does not reflect the inclusion of the Linux brand or trademark in the styx project.
Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.
Terms and Conditions (TODO: fix this!)