Contribute to Open Source. Search issue labels to find the right project for you!

Raw string literals

hngnaig/notes

raw string không xử lý bất kỳ escapes nào. Raw string được bắt đầu bằng ký tự r bắt buộc, theo sau đó ký tự # (U+0023) và double-quote " (U+0022), có thể có một hoặc nhiều #" liên tiếp nhau, và kết thúc "#, số ký tự ", # sẽ phụ thuộc vào số ký tự bắt đầu.

r#"foo"#; // foo
r#""foo""#; // "foo"
r####""""foo""""####; // """foo"""
r#"\x56"#; // \x56foo; not vfoo;
"\\x56"; // \x56
Updated 23/06/2017 10:25

Bảng escapes ở Rust

hngnaig/notes

Byte escapes

code Name
\x7F 8-bit character code (exactly 2 digits)
\n Newline
\r Carriage return
\t tab
\\ Backslash
\0 Null

Unicode escapes

Code Name
\u{7FFF} 24-bit unicode character code (up to 6 digits)

Quote escapes

Code Name
\' Single quote
\" Double quote
Updated 23/06/2017 09:42

Characters and strings literal

hngnaig/notes

Một vài cách khai báo với strings và characters rust // character let _ = 'H'; // string let _ = "string"; // raw let _ = r#"hello"#; // byte let _ = b'H'; // byte string let _ = b"hello"; // byte raw string // let _ = br#"hello"#;

Updated 23/06/2017 10:03

Escape \n String

hngnaig/notes

Khi bạn có một chuỗi (string), nếu bạn escape ký tự xuống dòng \n bằng cách thêm \ vào giữa string như sau: rust let b = "some\ thing"; thì trong chuỗi trên escape sẽ bị bỏ qua và sẽ bằng với chuỗi sau: ```rust let a = “something”;

assert_eq!(a, b); ```

Updated 23/06/2017 09:09

Decide whether to use Groth16 or PHGR13/BCTV14 for Sapling

zcash/zcash

PHGR13/BCTV14 is our current proving system. (See section 5.7 of the protocol spec and references there. The minor differences from the PHGR13 paper include adapting to Type 3 pairings, and a slight change to the verification key.)

Groth16 is a more recent proving system designed by Jens Groth and described in https://eprint.iacr.org/2016/260 . It has smaller proofs (about <sup>4</sup>/<sub>9</sub> the size of PHGR13/BCTV14 for a given curve), and faster proving and verification. It is proven secure in the Subversion Generic Bilinear Group Model (Sub-GBGM); see https://eprint.iacr.org/2017/599 . (The Sub-GBGM is a more realistic model than the GBGM used in Groth’s paper, so this is a stronger result than the original one by Groth. The changes made by Abdolmaleki et al are only to the CRS generation and verification, not to proving or verifying.)

Proofs in the Sub-GBGM model a Type 3 pairing abstractly, only allowing the adversary to use certain operations on group elements (group addition in G<sub>1</sub> or G<sub>2</sub>; multiplication in G<sub>T</sub>; pairing; picking a random G<sub>{1, 2, T}</sub> element with unknown discrete log; and comparison of group elements). Of course a real adversary is not limited to using these operations when manipulating group elements, so proofs in this model are only proofs of the difficulty of a certain class of generic attacks. The realism of the model is therefore an issue when assessing the value of the security proof, relative to security proofs based only on standard and/or knowledge assumptions which are available for PHGR13/BCTV14.

@ebfull has implemented Groth16 (several times! :-) ) in Rust. It’s likely that we would be using a Rust implementation if we switched to Groth16; also, it’s likely that we would be using the BLS12-381 curve. However we could also do each of these things (switch to Groth16, BLS12-381, or Rust) independently, so we need to consider the concrete performance improvement (and other features; see @ebfull’s comment below) of Groth16 while holding other factors constant, before making a decision about whether any risk associated with using the newer proving system is justified.

Updated 25/06/2017 19:14 17 Comments

Trait FromStr

hngnaig/notes
pub trait FromStr {
    type Err;
    fn from_str(s: &str) -> Result<Self, Self::Err> {}
}

Nếu implement trait này, chúng ta có thể convert một string nào đó sang type chúng ta muốn. Ví dụ:

let ip = "127.0.0.1".parse().unwrap();
println!("Debug: {:?}", ip);
/// IpAddress::V4("127.0.0.1")

hoặc: rust let two = "2".parse::<i32>().unwrap(); println!("{}", two); // 2 Hay nói cách khác thì i32 đã implement trait này như sau: ```rust impl FromStr for i32 { type Err = ParseErrror; fn from_str(s: &str) -> Result<i32, Self::Err> { unimplemted; } }

Updated 23/06/2017 09:15

Tạo document ở Rust

hngnaig/notes

Phần notes sẽ nói về comment và document.


Comment

grammar comment : block_comment | line_comment ; block_comment : "/*" block_comment_body * "*/" ; block_comment_body : [block_comment | character] * ; line_comment : "//" non_eol * ;

Example ```rust // line comment

/ block coment line 1 line 2 / ```

Updated 22/06/2017 14:08

tectonic 0.1.5 (new formula)

Homebrew/homebrew-core
  • [x] Have you followed the guidelines for contributing?
  • [x] Have you checked that there aren’t other open pull requests for the same formula update/change?
  • [x] Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you’re submitting?
  • [ ] Does your build pass brew audit --strict <formula> (after doing brew install <formula>)?

Thanks to #14490 getting merged in, we’re closer to a functional formula for Tectonic! I’m submitting this as a WIP to move Homebrew-specific discussion out of tectonic-typesetting/tectonic#4, but there’s a couple of things that need still need to be addressed:

  • [ ] Ideally, the test block should invoke tectonic. Unfortunately, a bunch of rather large TeX packages need to be downloaded in order to process even a minimal (empty) document.
  • [x] brew audit fails because the harfbuzz dependency needs to be installed with-graphite2.
Updated 27/06/2017 01:55 9 Comments

[query] Add type-hinting functions

mozilla/mentat

It would be convenient to have a way to constrain the types of variables explicitly:

[:find ?x :where
 [(uuid ?v)]
 [?x _ ?v]]

This hinting/constraining would be useful for queries like the above — find all entities associated with any UUID — but also allows for ground (and potentially other functions) to accept ambiguous EDN input. We don’t have a good way right now to distinguish between 10 the Long and 10 the entid.

These functions are very straightforward to implement.

Updated 16/06/2017 19:17 1 Comments

Port llvm-hs to Rust

glaebhoerl/personal

llvm-hs-pure -> llvm-ast llvm-hs -> ?

Depend on llvm-sys for FFI.

Use language-haskell-syntax and output raw Rust (as string) to automate bulk of grunt work?

AST: Box, Rc, Arc? Can we abstract it? (Fake HKTs with associated types? How about &'a T or Cow?) [a] -> Vec<T>? Or Cow<'a, [T]>?

https://hackage.haskell.org/package/llvm-hs https://hackage.haskell.org/package/llvm-hs-pure

Updated 23/05/2017 09:41

new verifier implemented in Rust

zcash/zcash

Write a new verifier for the Zcash v1 (“Sapling”) ZK-proofs from the ground up in Rust. Run both the current verifier unchanged and the new verifier in parallel so that it doesn’t take anymore time (typically), even though it takes more CPU. Wait til both verifiers have completed. If they both return true, then the combo returns true. If they both return false then the combo returns false. If they return different things than each other, then the combo logs a detailed error message (including the proof in question and the current message and so on) and then aborts the process.

(This aborting-the-process part raises questions about blockchain forks and forward progress, but I’m pretty sure it is far more important to alert users and developers and to prevent users from accidentally accepting invalid payments than it is to minimize chances of a blockchain fork. Also, as the Zcash employees well learned during the recent security incident (https://z.cash/blog/security-announcement-2017-04-13.html), I’m firmly of the opinion that the only way to communicate with most users is to abort the process.)

Updated 20/04/2017 22:15 9 Comments

[parser-utils] Determine a pattern for using `combine` to parse tuples

mozilla/mentat

This is follow-up to https://github.com/mozilla/mentat/pull/393#discussion_r109059135. It’s not easy to parse two streams in lock-step using combine; this would be handy for parsing maps, which iterate naturally as (k, v) pairs. In response I flatten into a sequence [k1 v1 k2 v2 ...], which combine handles just fine.

combine takes tuples to mean order: that is, (p1, p2) expects p1 to succeed (possibly consuming input) and then p2 to succeed, starting after the input that p1 consumed.

Suppose for the moment that instead (p1, p2) means to parse p1 on the first element of tuples and p2 on the second element of tuples. I don’t know how sensible this is; for example, what would (many(p1), p2) do? I’m filing this to have a place to leave thoughts as they come to us.

Updated 03/04/2017 18:22

[query] Intern values in query parser

mozilla/mentat

It’s currently difficult for us to add any kind of interning to the EDN parser: see https://github.com/kevinmehall/rust-peg/issues/84.

However, in #395 I’m about to make Variable and TypedValue wrappers around an Rc. We thus have an opportunity to intern query parts within the query parser itself, even if the EDN value stream itself contains duplicate strings

This has some immediate value: not only do we get cloneable ConjoiningClauses (and other consumers of Variable and TypedValue) — the point of #395 — but also as we drop the repeated [edn::Value] parser inputs we can prune some memory.

To do this involves maintaining state for the duration of our combine parse: probably a little struct around a few InternSet<PlainSymbol> and InternSet<String> instances.

This would be threaded into the top parser (Find::find), and then down into each parser it creates. I think the simplest way to do that — avoiding lifetime and mutability issues — is to wrap our interner in an Rc and pass it by cloning. (It’s theoretically possible to use a ThreadLocal for this, but global state is a bit of a downer.)

We’d discard the interner when we’re done with the parse. A future optimization is to keep it around….

Updated 03/05/2017 15:32 2 Comments

ensure all transitive rust dependencies are fetched via our depends system.

zcash/zcash

build fail with librustzcash while using a proxy. affected repository: https://github.com/rust-lang/crates.io-index

output: cd /home/riordant/Documents/Project/zcash/depends/work/build/x86_64-unknown-linux-gnu/librustzcash/0.1-2078cca28b9/.; PATH=/home/riordant/Documents/Project/zcash/depends/x86_64-unknown-linux-gnu/native/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin cargo build --release Updating registry `https://github.com/rust-lang/crates.io-index` warning: spurious network error (2 tries remaining): [12/-2] [7] Couldn't connect to server (Unsupported proxy scheme for 'https://www-proxy.scss.tcd.ie:8080/') warning: spurious network error (1 tries remaining): [12/-2] [7] Couldn't connect to server (Unsupported proxy scheme for 'https://www-proxy.scss.tcd.ie:8080/') error: failed to fetch `https://github.com/rust-lang/crates.io-index`

zcash version: v1.0.8

  • OS name + version: Ubuntu 16.04.2 (VM)
  • RAM: 8GB
  • Disk size: 39GB
  • Linux kernel version (uname -a):Linux riordant-virtual-machine 4.4.0-21-generic #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  • Compiler version (gcc -version): gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Updated 10/05/2017 15:23 6 Comments

[tx] Accept [a v] notation for lookup refs in entity position

mozilla/mentat

This is follow-up tracking https://github.com/mozilla/mentat/pull/382#pullrequestreview-28255925. In #382, I accept only (lookup-ref a v) for lookup-refs in both entity and value position (e and v in a [:db/add e _ v] assertion). That’s contrary to Datomic, which accepts embedded vectors like [a v] in both locations, treating ambiguity in some ad-hoc way. (See the discussion in #183 for notes on this ambiguity.)

This ticket tracks accepting an embedded vector [a v] in the entity position only. We suspect that the majority of lookup refs are entity refs in this form, and it’s both shorter to type and handy to agree with Datomic.

To implement this, you’ll need to:

1) verify that entid_or_lookup_ref_or_temp_id is still used only to parse entity positions; 2) add a vector_lookup_ref function accepting [a v] around https://github.com/mozilla/mentat/blob/8adb6d97fd55b262d7d89708db86eb37f0e5ab0d/tx-parser/src/lib.rs#L72; 3) use your new function around https://github.com/mozilla/mentat/blob/8adb6d97fd55b262d7d89708db86eb37f0e5ab0d/tx-parser/src/lib.rs#L88; 4) add tests demonstrating you can used nested vectors in entity position and that you can use nested vectors in value position – but they’re not interpreted as lookup refs. This latter may already be tested by the existing tests!

Updated 28/03/2017 16:11

[tx] Handling retracting schema characteristics, like `:db/cardinality` and `:db/unique`.

mozilla/mentat

After #370 (fixing #294 and #295), we still need to implement handling [:db/retract :attribute :db/* ...] of schema characteristics. This might also address [:db/retractEntity :attribute] if it comes after https://github.com/mozilla/mentat/issues/378.

The tricky thing to ensure here is that we fall back to the default correctly, and that the “schema” materialized view does not get out of sync with the in-memory state.

Updated 24/04/2017 18:09

[tx] Test that transaction IDs are in the correct partition, strictly monotonic increasing, etc

mozilla/mentat

This ticket tracks verifying that transaction IDs are: - in :db.part/tx; - strictly monotonically increasing; - preserved across DB close and re-open – this is really testing the partition map updating.

This is follow-up to https://github.com/mozilla/mentat/pull/364#discussion_r105274423.

We should also test that :db/txInstant are: - monotonically increasing; - in a valid range.

This is follow-up to https://github.com/mozilla/mentat/pull/364#discussion_r105274261.

It might be necessary to provide a clock in some form to transact, and to make the global Conn and the local TestConn provide clocks.

Updated 24/04/2017 18:09

Update FasterPath code for Windows compatibility

danielpclark/faster_path

Appveyor tests have been added and have shown 23 errors in FasterPath tests on Windows. This is without monkey patches or the WITH_REGRESSION environment flag.

Started with run options --seed 43247
AbsoluteTest
 FAIL (0.00s) :: test_it_determins_absolute_path
Expected false to be truthy.
C:/projects/faster-path/test/absolute_test.rb:9:in `test_it_determins_absolute_path'
 FAIL (0.00s) :: test_it_returns_similar_results_to_pathname_absolute?
Expected: true
Actual: false
C:/projects/faster-path/test/absolute_test.rb:15:in `block in test_it_returns_similar_results_to_pathname_absolute?'
C:/projects/faster-path/test/absolute_test.rb:14:in `each'
C:/projects/faster-path/test/absolute_test.rb:14:in `test_it_returns_similar_results_to_pathname_absolute?'
 PASS (0.00s) :: test_it_safely_takes_nil
DirnameTest
 PASS (0.00s) :: test_it_does_not_modify_its_argument
 FAIL (0.04s) :: test_it_returns_the_return_all_the_components_of_filename_except_the_last_one_unix_format
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: UTF-8
-"/"
+# encoding: ASCII-8BIT
+"."
C:/projects/faster-path/test/dirname_test.rb:30:in `test_it_returns_the_return_all_the_components_of_filename_except_the_last_one_unix_format'
 FAIL (0.01s) :: test_it_ignores_a_trailing_slash
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: UTF-8
-"/foo"
+# encoding: ASCII-8BIT
+"."
C:/projects/faster-path/test/dirname_test.rb:25:in `test_it_ignores_a_trailing_slash'
 PASS (0.00s) :: test_it_gets_dirnames_correctly
 FAIL (0.01s) :: test_it_returns_all_the_components_of_filename_except_the_last_one
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: UTF-8
-"/home"
+# encoding: ASCII-8BIT
+"."
C:/projects/faster-path/test/dirname_test.rb:5:in `test_it_returns_all_the_components_of_filename_except_the_last_one'
 FAIL (0.01s) :: test_it_returns_the_return_all_the_components_of_filename_except_the_last_one_edge_cases
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: UTF-8
-"./b"
+# encoding: ASCII-8BIT
+"."
C:/projects/faster-path/test/dirname_test.rb:40:in `test_it_returns_the_return_all_the_components_of_filename_except_the_last_one_edge_cases'
 PASS (0.00s) :: test_it_returns_a_string
DirectoryRefinementTest
 PASS (0.00s) :: test_refines_pathname_directory?
 PASS (0.00s) :: test_nil_behaves_the_same
DirectoryTest
 PASS (0.00s) :: test_of_Cs
 FAIL (0.01s) :: test_of_Bs
Expected: false
Actual: true
C:/projects/faster-path/test/directory_test.rb:42:in `test_of_Bs'
 PASS (0.00s) :: test_of_As
 PASS (0.00s) :: test_nil_for_directory?
RelativeRefinementTest
 PASS (0.00s) :: test_nil_behaves_the_same
 FAIL (0.00s) :: test_refines_pathname_relative?
Expected true to not be truthy.
C:/projects/faster-path/test/refinements/relative_test.rb:14:in `test_refines_pathname_relative?'
HasTrailingSeparatorTest
 FAIL (0.00s) :: test_has_trailing_separator_against_pathname_implementation
Expected: true
Actual: false
C:/projects/faster-path/test/has_trailing_separator_test.rb:17:in `test_has_trailing_separator_against_pathname_implementation'
 PASS (0.00s) :: test_has_trailing_separator?
 PASS (0.00s) :: test_has_trailing_separator_with_nil
HasTrailingSeparatorRefinementTest
 PASS (0.00s) :: test_refines_pathname_has_trailing_separator?
JoinTest
 PASS (0.00s) :: test_join
ChopBasenameTest
 FAIL (0.00s) :: test_of_Cs
Expected: ["http://", "www.example.com"]
Actual: ["", "http://www.example.com"]
C:/projects/faster-path/test/chop_basename_test.rb:130:in `test_of_Cs'
 PASS (0.00s) :: test_nil_inputs
 FAIL (0.00s) :: test_of_As
Expected: ["aa/a//", "a"]
Actual: ["", "aa/a//a"]
C:/projects/faster-path/test/chop_basename_test.rb:83:in `test_of_As'
 PASS (0.00s) :: test_it_fixes_blank_results_to_pathname_chop_basename
 FAIL (0.02s) :: test_it_returns_similar_results_to_pathname_chop_basename_for_dot_files
a: .hello/world/ and b: .
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: UTF-8
-".hello/world/"
+# encoding: ASCII-8BIT
+""
C:/projects/faster-path/test/chop_basename_test.rb:71:in `block (2 levels) in test_it_returns_similar_results_to_pathname_chop_basename_for_dot_files'
C:/projects/faster-path/test/chop_basename_test.rb:70:in `each'
C:/projects/faster-path/test/chop_basename_test.rb:70:in `block in test_it_returns_similar_results_to_pathname_chop_basename_for_dot_files'
C:/projects/faster-path/test/chop_basename_test.rb:67:in `each'
C:/projects/faster-path/test/chop_basename_test.rb:67:in `test_it_returns_similar_results_to_pathname_chop_basename_for_dot_files'
 FAIL (0.01s) :: test_it_chops_basename_
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: ASCII-8BIT
-"/hello/world.txt"
+# encoding: UTF-8
+"world.txt"
C:/projects/faster-path/test/chop_basename_test.rb:11:in `test_it_chops_basename_'
 FAIL (0.01s) :: test_it_returns_similar_results_to_pathname_chop_basename
a: hello/world/ and b: .
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: UTF-8
-"hello/world/"
+# encoding: ASCII-8BIT
+""
C:/projects/faster-path/test/chop_basename_test.rb:28:in `block (2 levels) in test_it_returns_similar_results_to_pathname_chop_basename'
C:/projects/faster-path/test/chop_basename_test.rb:27:in `each'
C:/projects/faster-path/test/chop_basename_test.rb:27:in `block in test_it_returns_similar_results_to_pathname_chop_basename'
C:/projects/faster-path/test/chop_basename_test.rb:24:in `each'
C:/projects/faster-path/test/chop_basename_test.rb:24:in `test_it_returns_similar_results_to_pathname_chop_basename'
 FAIL (0.00s) :: test_of_Bs
Expected: ["../", "."]
Actual: ["", ".././"]
C:/projects/faster-path/test/chop_basename_test.rb:106:in `test_of_Bs'
 PASS (0.00s) :: test_it_returns_similar_results_to_pathname_chop_basename_for_slash
DelTrailingSeparatorTest
 PASS (0.00s) :: test_del_trailing_separator
 PASS (0.00s) :: test_del_trailing_separator_in_dosish_context
InstallBuildTest
 PASS (0.14s) :: test_cache_file
 SKIP (0.10s) :: test_install_gem_in_bundler_vendor
 SKIP (0.10s) :: test_install_gem_in_rvm_gemset
BlankTest
 PASS (0.00s) :: test_it_is_blank?
AddTrailingSeparatorTest
 PASS (0.00s) :: test_refines_pathname_add_trailing_separator_in_dosish_context
 FAIL (0.02s) :: test_add_trailing_separator_against_pathname_implementation
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: UTF-8
-"hello/"
+# encoding: ASCII-8BIT
+"hello\\"
C:/projects/faster-path/test/add_trailing_separator_test.rb:43:in `test_add_trailing_separator_against_pathname_implementation'
 PASS (0.00s) :: test_it_handles_nil
 PASS (0.00s) :: test_refines_pathname_add_trailing_separator
 FAIL (0.01s) :: test_add_trailing_separator_in_dosish_context
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: ASCII-8BIT
-"A:\\"
+# encoding: UTF-8
+"A:/"
C:/projects/faster-path/test/add_trailing_separator_test.rb:23:in `test_add_trailing_separator_in_dosish_context'
 FAIL (0.01s) :: test_add_trailing_separator
--- expected
+++ actual
@@ -1,2 +1,2 @@
-# encoding: ASCII-8BIT
-"hello\\"
+# encoding: UTF-8
+"hello/"
C:/projects/faster-path/test/add_trailing_separator_test.rb:12:in `test_add_trailing_separator'
AbsoluteRefinementTest
 PASS (0.00s) :: test_nil_behaves_the_same
 FAIL (0.00s) :: test_refines_pathname_absolute?
Expected false to be truthy.
C:/projects/faster-path/test/refinements/absoulte_test.rb:14:in `test_refines_pathname_absolute?'
CleanpathConservativeTest
 PASS (0.00s) :: test_clean_conservative_dosish_stuff
 PASS (0.00s) :: test_clean_conservative_defaults
PbenchTest
 FAIL (0.19s) :: test_it_calculates_the_correct_percentage
Expected 40..60 to include -8.9.
C:/projects/faster-path/test/pbench_test.rb:7:in `test_it_calculates_the_correct_percentage'
ExtnameTest
 PASS (0.00s) :: test_extname
 PASS (0.00s) :: test_extname_edge_cases
 PASS (0.00s) :: test_nil_inputs
 FAIL (0.01s) :: test_substitutability_of_rust_and_ruby_impls
--- expected
+++ actual
@@ -1,2 +1,2 @@
# encoding: ASCII-8BIT
-""
+".bar/baz"
C:/projects/faster-path/test/extname_test.rb:43:in `test_substitutability_of_rust_and_ruby_impls'
EntriesTest
 PASS (0.00s) :: test_it_returns_similar_results_to_pathname_entries_as_strings
RelativeTest
 FAIL (0.00s) :: test_it_knows_its_relativeness
Expected true to not be truthy.
C:/projects/faster-path/test/relative_test.rb:9:in `test_it_knows_its_relativeness'
 FAIL (0.00s) :: test_it_knows_its_relativeness_in_dos_like_drive_letters
Expected true to not be truthy.
C:/projects/faster-path/test/relative_test.rb:26:in `test_it_knows_its_relativeness_in_dos_like_drive_letters'
 PASS (0.00s) :: test_it_takes_nil_safely
ChopBasenameRefinementTest
 PASS (0.00s) :: test_refines_pathname_chop_basename
MonkeyPatchesTest
 PASS (0.00s) :: test_it_redefines_relative?
 PASS (0.00s) :: test_it_redefines_has_trailing_separator
 PASS (0.00s) :: test_it_redefines_absolute?
 PASS (0.00s) :: test_it_redefines_add_trailing_separator
 PASS (0.00s) :: test_it_does_not_redefine_directory?
 PASS (0.00s) :: test_it_redefines_chop_basename
CleanpathAggressiveTest
 PASS (0.00s) :: test_plus
 PASS (0.00s) :: test_clean_aggressive_dosish_stuff
 PASS (0.00s) :: test_clean_aggresive_defaults
 PASS (0.00s) :: test_clean_aggressive_defaults
BasenameTest
 PASS (0.00s) :: test_it_does_the_same_as_file_basename
 PASS (0.00s) :: test_it_creates_basename_correctly
 PASS (0.00s) :: test_nil_inputs
FasterPathTest
 PASS (0.00s) :: test_it_build_linked_library
Finished in 0.73281s
72 tests, 302 assertions, 23 failures, 0 errors, 2 skips
Updated 31/05/2017 00:52 2 Comments

[db] Abstract over user_version to allow database sharing

mozilla/mentat

Mentat uses a single integer, the SQLite user version, to handle base storage schema versioning. At some point it might be useful to abstract this a little, allowing the use of a separate table (à la the “table table” in Firefox for iOS) or other mechanism. This would allow Mentat to cooperatively store data inside an existing SQLite database owned by a conventional consumer.

Updated 24/04/2017 18:14 1 Comments

[query] Implement simple aggregation

mozilla/mentat

We can support simple aggregation in our queries simply by creating appropriate GROUP BY clauses (see #311) and projecting aggregate expressions directly in SQL.

The usual aggregate function symbols map directly to SQL aggregate functions:

(def aggregate-functions
  {:avg :avg
   :count :count
   :max :max
   :min :min
   :sum :total
   })

and the ClojureScript implementation offers a lot of guidance.

Updated 16/06/2017 19:20

[core] Add transaction listener interface

mozilla/mentat

This is the Rust equivalent of #61 and Datomic’s Transaction Report Queue. The idea is to push transaction reports to a queue (or invoke a callback, or …) to allow the embedding application to react to specific types of transactions. There are lots of ways to achieve this, all with different trade-offs.

If we want to push https://github.com/mozilla/tofino/issues/1575 all the way across the line, we’ll need some form of transaction listener to push changes to the WebSocket clients.

This depends on #296.

Updated 24/04/2017 22:08 1 Comments

[core] Allow canceling/interrupting Mentat queries

mozilla/mentat

One of the things @rnewman’s considered is how to allow canceling or interrupting Mentat queries. He concludes that the best way to achieve this is to farm queries to separate threads. That’s because SQLite doesn’t allow to cancel running queries, so the way to achieve this is to just kill the thread and let SQLite’s atomic commit mechanism handle clean-up, etc.

This ticket tracks experimenting with this approach, working through whatever communciation/synchronization primitives are necessary, whatever thread pooling is desired, etc.

Updated 24/04/2017 22:08 2 Comments

[query] Type code expansion

mozilla/mentat

As we process a collection of patterns into a CC, we collect constraints. For example,

[?x ?a true]

yields a constraint like

ColumnConstraint::EqualsValue(d0_v, TypedValue::Boolean(true))

Some types (including boolean) share a SQLite value representation with other types — 1 could be a ref, an integer, an FTS string, or a boolean.

This ticket tracks expanding appropriate value_type_tag constraints out of a set of column constraints once all value constraints are known. We naturally must wait until all value constraints are known, because ?a might later be bound, which will eliminate the need to constrain value_type_tag.

It’s possible at this point for the space of acceptable type tags to not intersect: e.g., the query

[:find ?x :where
 [?x ?y true]
 [?z ?y ?x]]

where ?y must simultaneously be a ref-typed attribute and a boolean-typed attribute, which is impossible. This function should deduce that and call ConjoiningClauses.mark_known_empty.

See also #292.

Updated 03/05/2017 14:00 5 Comments

[tx] Test that datom flags are written to SQL store correctly

mozilla/mentat

This is follow-up to #267 and similar to #268. What we want is tests to ensure the datoms table gets the correct index_* flags set, based on the attribute flags. The datoms schema is around https://github.com/mozilla/mentat/blob/42580539b8a58f9d10e5891a22112f8c1b697bff/db/src/db.rs#L56.

To test this, use the existing transact code to transact assertions using some of the bootstrap attributes. Then use raw SQL to query the datoms table, following the examples in https://github.com/mozilla/mentat/blob/42580539b8a58f9d10e5891a22112f8c1b697bff/db/src/debug.rs. (You might add a helper there to produce EDN for the “full” or “raw” datoms, making this a little easier to test, and possibly helping later on when we want to compare other parts of the raw tables.)

The interesting cases are: - attribute is :db/index true => datom has index_avet = 1 - attribute is :db/valueType :db.type/ref => datom has index_vaet = 1 - attribute is :db/fulltext true => datom has index_fulltext = 1 Note: this can’t be tested yet, since fulltext datoms aren’t yet supported! - attribute is :db/unique :db.unique/value => datom has unique_value = 1

Check the positive and negative cases for each of those, and we’re good! This is a pretty good first bug, I think, since there’s lots of code similar to this in the tree already.

Updated 24/04/2017 22:15

[query] Parse more complex query patterns

mozilla/mentat

After #211 we support queries consisting of one or more data patterns: [?x :foo/bar ?y]. We need to grow more:

  • [x] not
  • [x] not-join
  • [x] and
  • [x] or
  • [x] or-join
  • [ ] Predicate expressions (part done)
  • [ ] Rule expressions

This is unlikely to be a good first bug.

Updated 28/04/2017 09:47

[meta] Full Datalog type support (instants, UUIDs, URIs)

mozilla/mentat

Over in #170, I simplified things by not including placeholders for Mentat value types I wasn’t going to support at the start: :db.type/instant, :db.type/uuid, and :db.type/uri. (We might consider supporting JSON and a binary blob type like :db.type/bytes as well.)

This ticket tracks supporting these types. Off the top, this will mean: 1. Adding new DB_TYPE_* values to the bootstrapper; 1. Bumping the SQL schema to accommodate the new idents; 1. Adding new ValueType cases; 1. Adding corresponding TypedValue cases; 1. Implementing the conversions to and from SQL; 1. testing the new types in the transactor and potentially in the query engine as well.

Updated 24/04/2017 22:14 2 Comments

[tx] Prepare and cache SQL statements for :db/add, :db/retract, etc

mozilla/mentat

The set of SQL statements for adding and retracting datoms is limited. We will achieve significant performance benefits by caching the compiled statements and merely re-binding them.

This ticket tracks using the rusqlite statement cache (if it’s appropriate), or implementing our own statement cache for the transactor statements. It’s worth noting that the transactor will be single-connection-on-single-thread, so the cache will not need to be sophisticated.

Updated 21/04/2017 18:24 2 Comments

[tx] Wrap positive integer entids in an Entid newtype

mozilla/mentat

This is follow-up to https://github.com/mozilla/mentat/pull/171#discussion_r96086384 and #170. Right now, I’m representing positive integer entids as i64, since SQLite doesn’t have an unsigned u64 type. It’s an interesting challenge to really represent entids as an Entid newtype that is really u63. When I started, I didn’t understand the rusqlite traits FromSql and ToSql; now that I do, this should be straight-forward. I’m filing this as a [good next bug], I guess, since it’s not trivial. Use OrderedFloat for inspiration and remember that it never makes sense to do arithmetic on entids.

As a side benefit/hindrance while testing: this might allow Mentat to control the introduction of entids. The Clojure version accepted any integer as an entid, which was handy for testing but did not agree with Datomic. We might be able to make the only way to allocate an entid be via a tempid, which has good correctness properties: it’ll be harder to make errors in transactions, and it’ll be harder to pollute a datom store with disconnected components.

Updated 09/06/2017 22:51 2 Comments

Spacemacs freezes while trying to start a line of comment in rust mode

syl20bnr/spacemacs

Description :octocat:

Spacemacs freezes while trying to start a line of comment in rust mode.

Reproduction guide :beetle:

  • Start Emacs
  • Open a buffer with .rs ending that enables rust mode, e.g. test.rs
  • Try to start a line of comment by typing two / characters

Observed behaviour: :eyes: :broken_heart: Emacs freezes with 100% CPU usage. Using C-G might get out of the freeze but it doesn’t succeed all the time.

Expected behaviour: :heart: :smile: The editing continues normally.

System Info :computer:

  • OS: gnu/linux
  • Emacs: 26.0.50.2
  • Spacemacs: 0.200.7
  • Spacemacs branch: develop (rev. 0c3afd5)
  • Graphic display: t
  • Distribution: spacemacs
  • Editing style: vim
  • Completion: helm
  • Layers: elisp (helm speed-reading auto-completion better-defaults spell-checking syntax-checking semantic rcirc git github version-control (shell :variables shell-default-height 30 shell-default-position 'bottom shell-default-term-shell "/bin/zsh" shell-enable-smart-eshell t) (ranger :variables ranger-show-preview t ranger-show-literal nil ranger-parent-depth 3 ranger-cleanup-eagerly t ranger-show-dotfiles t ranger-max-preview-size 10) (typography :variables typography-enable-typographic-editing t) wakatime chrome search-engine restclient dash fasd pandoc colors themes-megapack gtags imenu-list vim-powerline evil-snipe emacs-lisp fsharp (haskell :variables haskell-completion-backend 'intero haskell-enable-hindent-style "gibiansky") elixir org latex markdown clojure python html yaml shell-scripts sql javascript ruby (c-c :variables c-c -default-mode-for-headers 'c -mode c-c -enable-clang-support t) rust)
  • System configuration features: XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS NOTIFY LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 MODULES XWIDGETS LIBSYSTEMD

Backtrace :paw_prints:

Since Emacs freezes, I wasn’t able to get any backtrace.

Updated 20/06/2017 01:33 2 Comments

Rust backlog

mozilla/mentat

We’re starting on a Rust rewrite of the existing Clojure codebase, using the rust branch.

With an eye to parallelism, the following will get us to parity and a bit further, at which point we’ll start on the list of more speculative work.

Large portions of this will be easier than the first time around: the ClojureScript implementation required lots of deep thinking that won’t need to be repeated.

Starter Projects:

  • [x] Learning Rust. This is a totally valid use of time.
  • [x] [@bgrins] Get Rust builds and tests working in CI.
    • [x] Land initial code (https://github.com/mozilla/datomish/pull/134)
    • [x] Update readme with build and test instructions (https://github.com/mozilla/datomish/pull/135)
  • [x] [@jsantell] Begin SQLite wrapper
    • [x] Survey of Rust SQLite libraries.
    • [x] [@bgrins] dependency
    • [x] open a DB at a path and run the default pragmas
    • [x] exec/query abstractions
    • [x] How do we update and deliver SQLite?
  • [x] [@joewalker] Build an EDN parser.
  • [ ] [@jsantell] Build a Rust node native module. End goal: call a Rust function from JS.
  • [ ] Build a Rust library for Android. Call it via JNI.
  • [ ] Build a Rust library for iOS. Sign it. Call it from Swift.
  • [x] [@bgrins] Command-line tool. Right now it wouldn’t do much, but we could put the skeleton in place and do something like print user_version for a DB path… https://github.com/mozilla/datomish/pull/136
    • [x] Evaluate exposing as a crate within a subfolder
  • [x] [@bgrins] Figure out an out-of-band test solution for Rust. In-code tests are great, but they’re going to get really verbose when we have piles of SQL generation tests. (see https://github.com/mozilla/datomish/tree/rust/tests)
  • [ ] SQLite generator libraries a la honeysql: #273
    • [ ] Research.
    • [ ] Import and use.
  • [ ] Add a logger, something like https://github.com/slog-rs/slog

Moving on:

  • [x] Core (the big blocker for much of the rest):
    • [x] Core valueType work: type codes and translation, value conversion.
    • [x] Core schema work. Basic Rust type definitions. Traits for translations.
    • [x] SQLite wrapper: opener, part reading, etc.
    • [x] Bootstrapper.
    • [x] Conn + DB.
  • [ ] Querying:
    • [x] [@rnewman] Datalog query parser: port from DataScript. #160.
    • [x] Parsed query (FindRel) -> abstract query (ConjoiningClauses). Porting datomish.query.
      • This will involve building up much of ‘Core’, above, brick by brick.
    • [x] Abstract query -> SQL.
    • [x] Abstract query -> projectors.
    • [ ] Querying history and logs.
  • [ ] Transactor:
    • [x] Transactor expansion: schema + shorthand -> datoms.
    • [x] Transactor writer.
    • [ ] Transactor loop.
    • [x] Prepared statements/statement caching.
    • [ ] Excision.
    • [ ] noHistory.
  • [ ] Schema management module (porting).
  • [ ] Reader pool (query executor).
    • [ ] Query interruption via closing connection.
    • [ ] Parallel reads.
    • [ ] Connection affinity to exploit sqlite page cache.
    • [ ] Prepared statements for repeated queries.
      • [ ] Invalidation when schema contents change.
      • [ ] Handling of query transformation in the presence of bound values, which might change between uses of the prepared statement.
  • [ ] Tooling. We can invest in this as much as we want; it’s leverage.
    • [ ] A command-line query tool, just like sqlite3.
    • [ ] A REPL that’s capable of doing programmatic read/write tasks. JS bindings? Rust REPL?
    • [ ] A developer tool/embedded server: showing query translations and plans, timing query execution, exploring schema…
    • [ ] GraphQL interface generation from Datomish schema.
    • [ ] Bulk loading and import. Make this independent by expanding into a datom stream wrt a given schema.
    • [ ] Export: textual/EDN, RDF?
  • [ ] Transaction listeners over a socket.
    • [ ] Log dumper standalone tool. Watch writes happening as they occur.
  • [ ] Syncing:
    • [ ] Transaction log replication (out) and replay (in).
      • [ ] Code and algorithms exist.
      • [ ] Define how IDs are mapped in and out, and how those might be efficiently represented in chunks.
      • [ ] Handle application of noHistory attributes.
      • [ ] Handle remote excision.
    • [ ] Multi-master transaction log handling (syncing).
Updated 13/04/2017 02:28 2 Comments

Easy methods to start learning Rust with

danielpclark/faster_path

If you see a method you would like to implement open a new issue and get started with the outline below.

Good starter methods

  • <strike>Pathname#has_trailing_separator?</strike>
  • <strike>Pathname#add_trailing_separator</strike>
  • Pathname#del_trailing_separator
  • <strike>Pathname#extname</strike>

    Outline

  • Write Rust implementation of Pathname to FasterPath
  • Write refinements and monkey-patches for Pathname and map implementation to FasterPath
  • Write tests proving FasterPath operates the same as Pathname
  • Write benchmarks in test/benches and compare the performance. And make a copy of the tests for test/pbench in the pbench_suite.rb file (the outline is in the file).
  • Also compare both the original to the new method to verify compatibility. (See chop_basename A,B,C tests)
Updated 31/05/2017 05:50 4 Comments

Implement Pathname#join

danielpclark/faster_path

Rust seems to have it’s own path joining methods. http://rustbyexample.com/std_misc/path.html This “may” do exactly what we want. - [ ] Write Rust implementation of Pathname#join to FasterPath.join - [ ] Write refinements and monkey-patches for Pathname#join and map implementation to Pathname.new(result) - [ ] Write tests proving FasterPath.join operates the same as Pathname#join - [ ] Write benchmarks in test/benches and compare the performance. - [ ] Copy benchmarks to test/pbench/pbench_suite.rb

Updated 31/05/2017 05:50 2 Comments

Implement Pathname#plus

danielpclark/faster_path

This is a rather long method and highly used one. It would be great to have a Rust implementation of it. Here’s the current Ruby source:

def plus(path1, path2) # -> path # :nodoc:
  prefix2 = path2
  index_list2 = []
  basename_list2 = []
  while r2 = chop_basename(prefix2)
    prefix2, basename2 = r2
    index_list2.unshift prefix2.length
    basename_list2.unshift basename2
  end
  return path2 if prefix2 != ''
  prefix1 = path1
  while true
    while !basename_list2.empty? && basename_list2.first == '.'
      index_list2.shift
      basename_list2.shift
    end
    break unless r1 = chop_basename(prefix1)
    prefix1, basename1 = r1
   next if basename1 == '.'
    if basename1 == '..' || basename_list2.empty? || basename_list2.first != '..'
      prefix1 = prefix1 + basename1
      break
   end
    index_list2.shift
    basename_list2.shift
  end
  r1 = chop_basename(prefix1)
  if !r1 && /#{SEPARATOR_PAT}/o =~ File.basename(prefix1)
    while !basename_list2.empty? && basename_list2.first == '..'
      index_list2.shift
      basename_list2.shift
    end
  end
  if !basename_list2.empty?
    suffix2 = path2[index_list2.first..-1]
    r1 ? File.join(prefix1, suffix2) : prefix1 + suffix2
  else
    r1 ? prefix1 : File.dirname(prefix1)
  end
end
private :plus

At the moment I’m not using regex in the library so it would be great if we could implement if !r1 && /#{SEPARATOR_PAT}/o =~ File.basename(prefix1) with something like str.contains(MAIN_SEPARATOR).

Better yet if you look into Rust’s path implementation you may find something that does all of this for you. We’ll need to have tests to confirm the same behavior is achieved.

Updated 31/05/2017 05:50 5 Comments

Fork me on GitHub