Contribute to Open Source. Search issue labels to find the right project for you!

error running record_multi_step.py on new_collision

SCAII/SCAII
(C:\Program Files\Anaconda3) C:\Users\Jedi\.scaii\git\SCAII\backends\sky-rts\demos>python record_multi_step.py
rts got Cfg
Possible reward types: {'took_damage', 'kill_bonus', 'friendly_kill', 'damaged_enemy', 'harmed_friend'}
Possible actions: {'actions': {'top_left': 4, 'bottom_right': 1, 'bottom_left': 3, 'top_right': 2}, 'desc': 'Use action.attack_quadrant(1-4) to select a quadrant to attack'}
Action description MoveList -- unit ID, command, and target
rts got ResetEnv
Collision event: Started(CollisionObjectHandle(0), CollisionObjectHandle(1)) :::  Data1: ColliderData { e: Entity(0, Generation(1)), owner: None, sensor: false } Data2: ColliderData { e: Entity(1, Generation(1)), owner: Some(Entity(0, Generation(1))), sensor: true }
Collision event: Started(CollisionObjectHandle(6), CollisionObjectHandle(7)) :::  Data1: ColliderData { e: Entity(6, Generation(1)), owner: None, sensor: false } Data2: ColliderData { e: Entity(7, Generation(1)), owner: Some(Entity(6, Generation(1))), sensor: true }
Collision event: Started(CollisionObjectHandle(8), CollisionObjectHandle(9)) :::  Data1: ColliderData { e: Entity(8, Generation(1)), owner: None, sensor: false } Data2: ColliderData { e: Entity(9, Generation(1)), owner: Some(Entity(8, Generation(1))), sensor: true }
Collision event: Started(CollisionObjectHandle(2), CollisionObjectHandle(3)) :::  Data1: ColliderData { e: Entity(2, Generation(1)), owner: None, sensor: false } Data2: ColliderData { e: Entity(3, Generation(1)), owner: Some(Entity(2, Generation(1))), sensor: true }
Collision event: Started(CollisionObjectHandle(4), CollisionObjectHandle(5)) :::  Data1: ColliderData { e: Entity(4, Generation(1)), owner: None, sensor: false } Data2: ColliderData { e: Entity(5, Generation(1)), owner: Some(Entity(4, Generation(1))), sensor: true }
recorderConfig pkt : ScaiiPacket { src: Endpoint { endpoint: Some(Agent(AgentEndpoint)) }, dest: Endpoint { endpoint: Some(Recorder(RecorderEndpoint)) }, specific_msg: Some(RecorderConfig(RecorderConfig { pkts: [ScaiiPacket { src: Endpoint { endpoint: Some(Agent(AgentEndpoint)) }, dest: Endpoint { endpoint: Some(Core(CoreEndpoint)) }, specific_msg: Some(Config(Cfg { which_module: Some(CoreCfg(CoreCfg { plugin_type: PluginType { plugin_type: Some(SkyRts(SkyRts)) } })) })) }, ScaiiPacket { src: Endpoint { endpoint: Some(Agent(AgentEndpoint)) }, dest: Endpoint { endpoint: Some(Backend(BackendEndpoint)) }, specific_msg: Some(Config(Cfg { which_module: Some(BackendCfg(BackendCfg { cfg_msg: Some([10, 12, 10, 10, 109, 117, 108, 116, 105, 95, 115, 116, 101, 112]), is_replay_mode: false })) })) }], overwrite: false, filepath: None })) }
...........persisting HEADER Header(ReplayHeader { configs: SerializedProtosScaiiPacket { data: [122, 56, 10, 18, 42, 6, 10, 4, 10, 2, 10, 0, 242, 1, 2, 26, 0, 250, 1, 2, 18, 0, 10, 32, 42, 20, 18, 18, 10, 14, 10, 12, 10, 10, 109, 117, 108, 116, 105, 95, 115, 116, 101, 112, 16, 0, 242, 1, 2, 10, 0, 250, 1, 2, 18, 0, 16, 0, 242, 1, 2, 50, 0, 250, 1, 2, 18, 0] } })
[0 1 2]
0
0
acting
RECORDER: action has expl_point?  YES
.............persisting KEYFRAME
Action caused rts.update()
Collision event: Started(CollisionObjectHandle(0), CollisionObjectHandle(2)) :::  Data1: ColliderData { e: Entity(0, Generation(1)), owner: None, sensor: false } Data2: ColliderData { e: Entity(2, Generation(1)), owner: None, sensor: false }
Collision event: Started(CollisionObjectHandle(1), CollisionObjectHandle(2)) :::  Data1: ColliderData { e: Entity(1, Generation(1)), owner: Some(Entity(0, Generation(1))), sensor: true } Data2: ColliderData { e: Entity(2, Generation(1)), owner: None, sensor: false }
Collision event: Started(CollisionObjectHandle(0), CollisionObjectHandle(3)) :::  Data1: ColliderData { e: Entity(0, Generation(1)), owner: None, sensor: false } Data2: ColliderData { e: Entity(3, Generation(1)), owner: Some(Entity(2, Generation(1))), sensor: true }
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
.............persisting KEYFRAME
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
RECORDER: action has expl_point?  NO
...........persisting DELTA
[0]
Traceback (most recent call last):
  File "record_multi_step.py", line 67, in <module>
    s = env.act(act, explanation=explanation)
  File "C:\Users\Jedi\.scaii\glue\python\scaii\env\__init__.py", line 302, in act
    resp = self._decode_handle_msg()
  File "C:\Users\Jedi\.scaii\glue\python\scaii\env\__init__.py", line 436, in _decode_handle_msg
    env_state=secret_state
  File "C:\Users\Jedi\.scaii\glue\python\scaii\env\sky_rts\env\state.py", line 36, in __init__
    faction_ids, np.unique(faction_ids)).astype(np.float)[:, :, 1:]
ValueError: could not broadcast input array from shape (40,40,0) into shape (40,40,2)
Updated 23/05/2018 20:35 1 Comments

Upgrade specs

SCAII/SCAII

I was trying to avoid this, but it needs to be done. We’ve hit the absolute limit of number of components that can be serialized with the current system. This is fixed in the new version, but not in a way I can just copy+paste and backport.

I did a minor update to give us a couple more, but when I tried to add any more than that I got compiler errors.

Updated 23/05/2018 08:00

Time-lapse triggers

SCAII/SCAII

Allow global (or maybe later, unit-based) triggers to operate on a specific timer pulse to call a certain Lua trigger. Essentially an extension of #103 but with time instead of space.

Updated 18/05/2018 20:55

Tracking Issue for Tower Defense

SCAII/SCAII
  • [ ] Pass 1 (1 day)
    • [x] Unwrap map of info into hover box (#99, #100)
    • [ ] Straight path for one triangle
      • [ ] Issue unit orders from Lua (#101) (0.5 day)
      • [x] Global/entity data store (#102, #110)
      • [ ] Trigger volume for agent enter/exit triggers (#103) (0.5 day after display bug fixed)
        • [x] Collision Tweak - collision should be propagated rather than immediately handled
        • [x] Fix tower attack bug - towers stop firing when unit leaves attack radius (#111)
  • [ ] Pass 2 (1 day)
    • [ ] Make path crooked w/ trigger volume and unit orders
    • [ ] Render landscapes or “decorative” objects (#104)
  • [ ] Pass 3 (2-3 days)
    • [ ] Place agent towers
      • [ ] Fix Python state representation to allow arbitrary data (#106) (~2 day)
      • [ ] Allow agent to build/spawn units (#107) (0.5 day)
        • [ ] Allow Lua to interpret/translate actions ? (#105) (? 0.5 at most)
  • [ ] PASS 4 (1.5 day)
    • [ ] Add Multiple unit types including rock paper scissors (0.5 day)
      • [ ] Unit armor/damage types (#108)
    • [ ] Timer trigger (#113) (Essentially free, 0.25 day at most)
    • [ ] Allow agent to upgrade towers (0.5 day, less if Lua actions done is pass 3)
      • May be easiest with #105
      • Should these be separate unit types, or an upgrade field?
      • May end up involving a version of #109 but with the glue side
Updated 23/05/2018 20:20

Consider communicating Lua data to visualization

SCAII/SCAII

This has two related pieces:

  1. It would be useful to communicate all Lua persistent data (#102, #110) to the visualization in a debug case (relies on #100), so we can see values changing frame to frame for the purpose of debugging inconsistencies.
  2. It would also be useful to communicate specific Lua globals all the time if they’re important to the scenario, this requires marking data as “debug” or “important” (names need work) and only communicating one unless in debug mode.
Updated 25/05/2018 22:27 5 Comments

Unit armor types

SCAII/SCAII

Units should have typed damage/damage resistance. However, I’m not certain whether this should be an intrinsic aspect of the engine, or naturally handled by Global Data Stores (#102) with an extension for execute-on-attack intrinsic triggers. If it’s not intrinsic, we should really consider #109 in order to communicate Lua-based concepts to the visualization end.

We may also need to communicate this as a feature layer in the future (similar to #109 but with the glue), but for now we’ll assume it’s part of the UnitType.

Updated 18/05/2018 20:54

Expand python state representation

SCAII/SCAII

Right now in Python we’re basically hardcoded for the tower scenario. The python side attempts to broadcast the state matrix we provide from Rust into a known form with a known number of layers. We could band-aid some of this by allowing the RTS-side to configure things in Python such as the expected values for factions, unit types, etc.

This would likely require specifying things like number of factions, etc in the Lua script.

However, it would probably be the most flexible and lucrative to communicate arbitrary state data to Python and allow the game wrapper to assemble it.

Updated 24/05/2018 21:47 2 Comments

Consider allowing Lua to interpret action spaces

SCAII/SCAII

It may be useful to allow Lua to interpret action commands. This would entail Lua both accepting and providing a list of proscribed action formats, including perhaps custom action types in the action list.

This would lighten the load on the glue side to find a way to interpret actions solely as move lists, as well as prevent weird hacks just to issue certain orders, and preventing the need to expose every bit of functionality while expecting the agent to “be honest” about only doing things that are allowed or expecting the Lua script to explicitly prohibit behavior.

Updated 18/05/2018 20:55

Allow non-interactable entities in RTS and Viz

SCAII/SCAII

Currently the framework technically exists for non-interactable objects in the RTS, but there’s no way to create them without direct access to the engine (manually create an entity with a renderable component but no attack data, HP, etc). We just need to allow a few Lua keys for this.

This also depends on marking things as “uninteractable” in the protobuf so that we know not to render healthbars, explanations, make it clickable etc. This also entails respecting layer information when rendering.

Updated 18/05/2018 20:11

RTS trigger volumes

SCAII/SCAII

Add collision of entities with “trigger volumes” that execute some Lua function with some arguments (including the global store, #102). For now, we won’t allow filtering based on criteria and expect Lua to return without changes otherwise.

This will allow us to do things like:

  • Delete or kill units that enter a certain region
  • Cause victory or defeat if a unit hits a goal
  • Spawn or recruit units if a unit enters a certain area

This will also implicitly set up a more general framework we can copy/rework/tweak later for things like time-based triggers or unit-ability based triggers.

Updated 21/05/2018 11:25 2 Comments

RustBelt: Securing the Foundations of the Rust Programming Language

Centril/interesting-papers

Paper hosted by author DOI (10.1145/3158154)

Abstract

Rust is a new systems programming language that promises to overcome the seemingly fundamental tradeoff between high-level safety guarantees and low-level control over resource management. Unfortunately, none of Rust’s safety claims have been formally proven, and there is good reason to question whether they actually hold. Specifically, Rust employs a strong, ownership-based type system, but then extends the expressive power of this core type system through libraries that internally use unsafe features. In this paper, we give the first formal (and machine-checked) safety proof for a language representing a realistic subset of Rust. Our proof is extensible in the sense that, for each new Rust library that uses unsafe features, we can say what verification condition it must satisfy in order for it to be deemed a safe extension to the language. We have carried out this verification for some of the most important libraries that are used throughout the Rust ecosystem.

Updated 04/05/2018 00:25

RTS Pathfinding

SCAII/SCAII

Below is a reproduction of SCAII/Sky-RTS#8 from before the merge:


Steering behaviors were removed due to some bugs, but we should add them back in soon. It’s likely best to use “steering whiskers” like we have “attack whiskers” that allow the unit to sense various units around it to apply forces.

Updated 03/05/2018 06:25

RTS Optimization Pass #1

SCAII/SCAII

This isn’t as much of an issue as before but worth keeping around. Following is a reproduction of SCAII/Sky-RTS#10:


We need to aim for much higher performance. At the moment we can only render ~30fps with state information AT FULL BLAST which is horrible. The following is a list of things to try, with notes:

  1. Change the State system to use better software rendering techniques. At the moment we just iterate over every pixel and check what’s there using collision. This is… bad. There are a few passes we could make

    1. Render entities rather than iterating over pixels. (Still requires clearing the board – slow)
    2. Use some simple techniques like Bresenham and FloodFill to render said entities on the board (faster than collision polling)
    3. Unrender and re-render only entities that have moved or been deleted (flood fill with zeroes)
  2. DRASTIC – modify core and glue to pass pointers straight into python and rely on never cloning the state packet.

  3. LESS DRASTIC BUT STILL TIME CONSUMING – Modify the protobuf to send information akin to what Viz currently does, and reinterpret it in Python. Doable, but annoying. Has some other major benefits. This is what SC2 does. We can interpret every unit as a circle to speed up rendering on the python side. This may end up biting us when we’re essentially doing the same thing in Python but slower, because Python. However, we make major gains in protobuf encoding/decoding speeds.

  4. Downsize all objects in collision to be balls under the hood; store in a KDTree and just do radius testing for ranges and collision. We’d still RENDER other shapes but under the hood they’re just circles. This also ditches the ncollide dependency which I think is probably too slow

  5. Possibly parallelize some things? This is undesirable because it just moves the problem elsewhere if someone wants to run multiple simulators in parallel.

  6. Do a pass at specs, consider only using thread local and such.

  7. DRASTIC – Ditch specs, it could be causing issues. This likely wouldn’t be TOO hard (just change SystemData to SystemArgs and you can manually pass references easily, the handling logic would be verbose though), but it’s a huge structure movement.

  8. Maybe some Lua tweaks?

  9. Give up and only use SC2 forever

Updated 02/05/2018 23:18

[Project Proposal] Add Stellar tipping to Meetup Bot

SFCryptocurrencyDevs/tipmebch

Let’s add stellar tipping to the rust meetup bot!

https://media.giphy.com/media/HtaGVNHVnTNuw/giphy.gif

Expected:

/tip 0.01 XLM rob

Bot: You have tipped 0.01 XLM to rob.

Commands:

/tip
/deposit
/balance

How it works

We will use a base account for receiving XLM and the memo field for determining who (meetup id) is depositing XLM. Then users can tip ANYONE in meetup with XLM, since we are just mapping balances to meetup id’s in our database. When a user wants to withdraw, they will simply ask the bot to withdraw the XLM from their account, specifying a stellar account id (public key) to send the XLM to – the bot will then subtract the XLM from their account in the database.

Steps

  • [x] Poll base account every X second(s), seeing if any new transactions were received (use cursor to keep track of “last seen” transaction)
  • [x] Remove unsafe in stellar polling
  • [x] Parse incoming transactions
  • [x] Setup server to run bot – hyper or rocket???
  • [ ] Setup postgres
  • [ ] Connect rust to postgres
  • [ ] Add new slash commands
  • [ ] Update accounts in db for X user with Y XLM balance

How to contribute?

Comment on this issue, describing what you are interested in building.

Updated 09/05/2018 07:02 2 Comments

Underscore Imports not expected

intellij-rust/intellij-rust
#![feature(underscore_imports)] // #48216

// std_prelude inspired
pub use std::{
    // Traits
    cmp::{Ord, PartialOrd},
    fmt::{Debug, Display, Write as _},
    io::{Read as _, Write as _, BufRead as _, Seek as _},
    hash::Hash,
};

Imgur

rust-lang/rust#48216: Tracking issue for RFC #2166: impl-only-use

Note that now rustc treats _ as a reserved identifier internally. (Logically it still isn’t – for example macro m($i:ident) {} m!(_) is still an error – but this simplifies the internals some.)

Updated 05/05/2018 11:11

Implement sapling as rust library for transaction generation and validation

zcash/zcash

Zcash currently has challenges on the horizon with cross platform support, lite clients, and network privacy, as well as the maintainability of a fork of the Bitcoin codebase. Looking longer term, there is the possibility that zcash switches codebases or changes consensus mechanisms. At the same time with sapling’s new crypto, the transaction generation and validation code needs a near complete rewrite for the new protocol. This is an opportunity.

Solutions to the above issues would all be considerably easier if all of the code necessary to generate transactions, including maintaining a copy of the merkle tree and “wallet” code that stores, selects, and assembles notes as inputs to a transaction , was in a separate library distinct from networking and “consensus facing code” (even though tx validation is part of consensus and probably should be a part of the library).

This would make it easy to support lite clients and to switch codebases. It would also make it easier to later produce software that submitted transactions via a mechanism that never touches the P2P network. This is important because both passive and active attacks can fingerprint the P2P network in ways that persist even if connections are over Tor.

It would also allow for improved security by keeping this library in a separate address space or other such countermeasures.

Since the proof generation code is already in rust, the principle code that needs to be redone is for the merkle tree and a “wallet” code

Updated 22/04/2018 18:15 6 Comments

Bridge continiously sending transactions with 'eth_sendTransaction timed out' message during stress testing

poanetwork/poa-bridge

This is the issue is the same as https://github.com/paritytech/parity-bridge/issues/149 but behavior is even worse due to automatic bridge restart implemented in POA bridge.

Network setup: * Home: a PoA testnet (Sokol) * Foreign: Ropsten

There are 1200 deposit transactions sent successfully to HomeBridge contract by a special python script. It took 8 blocks to validate all transactions. https://sokol-explorer.poa.network/block/1418808 … https://sokol-explorer.poa.network/block/1418815

The bridge discovered part of these transactions, tried to relay some of them, lost the connection and restarted: INFO:bridge::bridge::deposit_relay: got 7 new deposits to relay INFO:bridge::bridge::deposit_relay: relaying 7 deposits INFO:bridge::bridge::deposit_relay: deposit relay completed INFO:bridge::bridge::deposit_relay: got 129 new deposits to relay INFO:bridge::bridge::deposit_relay: relaying 129 deposits INFO:bridge::bridge::deposit_relay: deposit relay completed INFO:bridge::bridge::withdraw_relay: got 0 new signed withdraws to relay INFO:bridge::bridge::withdraw_relay: fetching messages and signatures INFO:bridge::bridge::withdraw_relay: fetching messages and signatures complete INFO:bridge::bridge::withdraw_relay: relaying 0 withdraws INFO:bridge::bridge::withdraw_relay: relaying withdraws complete INFO:bridge::bridge::withdraw_relay: waiting for signed withdraws to relay INFO:bridge::bridge::withdraw_confirm: got 0 new withdraws to sign INFO:bridge::bridge::withdraw_confirm: signing INFO:bridge::bridge::withdraw_confirm: signing complete INFO:bridge::bridge::withdraw_confirm: submitting 0 signatures INFO:bridge::bridge::withdraw_confirm: submitting signatures complete INFO:bridge::bridge::withdraw_confirm: waiting for new withdraws that should get signed INFO:bridge::bridge::deposit_relay: got 113 new deposits to relay INFO:bridge::bridge::deposit_relay: relaying 113 deposits INFO:bridge::bridge::deposit_relay: deposit relay completed INFO:bridge::bridge::deposit_relay: got 710 new deposits to relay INFO:bridge::bridge::deposit_relay: relaying 710 deposits WARN:bridge: Bridge is down with Request eth_sendTransaction timed out, attempting to restart WARN:<unknown>: Sending a response to deallocated channel: Ok([Ok(String("0xc47e8427c9eda913b9749bf0904cd8765b39f5448368194cb7aa4330bbe6a44d"))]) WARN:<unknown>: Sending a response to deallocated channel: Ok([Ok(String("0x4d96f7f39c50dd7b19aac9af708bca450fb6c9fb6b72024c7736d9883614845d"))]) ... WARN:<unknown>: Sending a response to deallocated channel: Ok([Ok(String("0x6b16c0ef9e34d65c4431c4e1e8c043564d53cf3ce440d7ea706164838904e209"))]) The database file was not updated so the restart of the bridge thread caused the same error. INFO:bridge::bridge::deposit_relay: got 951 new deposits to relay INFO:bridge::bridge::deposit_relay: relaying 951 deposits WARN:<unknown>: Sending a response to deallocated channel: Ok([Ok(String("0x9b034a82ebf6306933a35848f9d7b1552ababb8be5774e5501e4802013911a01"))]) WARN:<unknown>: Sending a response to deallocated channel: Ok([Ok(String("0x606b57af0b13dbeddf6b4c16833b0365dace777cc4feee82d2de14c5dde53c45"))]) So, the bridge is continuing to send transactions forever.

Even if the bridge process is killed manually, it is necessary to do manual modification of database but it causes lock of funds on HomeBridge contract side since incomplete amount of tokens is transfered by ForeignBridge contract.

Updated 20/03/2018 09:51 11 Comments

Bridge duplicates transactions after re-establishing connection to Parity

poanetwork/poa-bridge

The behaviour to re-establish IPC connection after parity re-run was introduced as part of fix for #22 (https://github.com/poanetwork/parity-bridge/commit/9c375cc89b810497cd700e73e39b89b0ab43c927).

But it caused appearance of minor regression: last set of transactions are being sent twice. Here is the logs from the foreign parity instance before it is killed: 2018-03-10 09:02:53 0/25 peers 87 KiB chain 5 MiB db 0 bytes queue 448 bytes sync RPC: 0 conn, 1 req/s, 175 µs 2018-03-10 09:03:10 Transaction mined (hash e0ac09e000484637fb5af07649d90103edbe6dc48640caa2e81956ac11d88b36) |-------- deposit() invocation 2018-03-10 09:03:10 Imported #6501 fa4a…cf4e (1 txs, 0.08 Mgas, 0.99 ms, 0.77 KiB) 2018-03-10 09:03:20 Transaction mined (hash 7914bbbcc10c105823689628b9815c340eb1a923433b8e5c64ebd8664df31eeb) |-------- approveAndCall() invocation 2018-03-10 09:03:20 Imported #6502 f517…7239 (1 txs, 0.06 Mgas, 1.01 ms, 0.83 KiB) 2018-03-10 09:03:25 Transaction mined (hash 2c6324d205e276170013f8f4450a7379998b4b4fb4b06001d3b296257b84e103) |-------- submitSignature() invocation 2018-03-10 09:03:25 Imported #6503 50b8…91df (1 txs, 0.26 Mgas, 1.12 ms, 1.03 KiB) 2018-03-10 09:03:28 0/25 peers 98 KiB chain 5 MiB db 0 bytes queue 448 bytes sync RPC: 0 conn, 2 req/s, 186 µs

The database file contains the following after that: $ cat ../erc20_db.toml home_contract_address = "0x2ace2268ed7a96713e6cd18e9b2c2ef0c306c22c" foreign_contract_address = "0x5e702ea5d81e14ba807fdd0ac923e811a12bfef1" home_deploy = 6209 foreign_deploy = 6488 checked_deposit_relay = 6218 checked_withdraw_relay = 6503 checked_withdraw_confirm = 6503

As soon as the parity instance is killed and run it produces the following logs: 2018-03-10 09:04:29 Starting Parity/v1.9.2-unstable-0feb0bb-20180201/x86_64-linux-gnu/rustc1.20.0 2018-03-10 09:04:29 Keys path /home/koal/parity/keys/PoA_foreign 2018-03-10 09:04:29 DB path /home/koal/parity/PoA_foreign/chains/PoA_foreign/db/d87bc73279457e60 2018-03-10 09:04:29 Path to dapps /home/koal/parity/PoA_foreign/dapps 2018-03-10 09:04:29 State DB configuration: fast 2018-03-10 09:04:29 Operating mode: active 2018-03-10 09:04:29 Configured for PoA_foreign using AuthorityRound engine 2018-03-10 09:04:30 Public node URL: enode://96b8fa10c0504cb3edb182786fcc8baf6cec3ff5bc4b5f9f82a24c49fe6bd5d09b64a8a2a595ec8d215d0523ec254cf31dda6151cf0a4669edc7bb964fd50af8@192.168.1.15:30343 2018-03-10 09:04:35 Transaction mined (hash 2faba8376df4632da7b4a0ce4cb984b7e51cc90849bf94e8a56a635195253537) |-------- deposit() invocation 2018-03-10 09:04:35 Transaction mined (hash 00da53a569c639617e23854a2fb3e80745690c69230e07df92e7159f1fe1c958) |-------- submitSignature() invocation 2018-03-10 09:04:35 Imported #6504 783d…e97e (2 txs, 4.20 Mgas, 1.15 ms, 1.23 KiB)

and the database file contains: $ cat ../erc20_db.toml home_contract_address = "0x2ace2268ed7a96713e6cd18e9b2c2ef0c306c22c" foreign_contract_address = "0x5e702ea5d81e14ba807fdd0ac923e811a12bfef1" home_deploy = 6209 foreign_deploy = 6488 checked_deposit_relay = 6219 checked_withdraw_relay = 6504 checked_withdraw_confirm = 6504

These transactions are handled properly by the contracts that’s why double-spend does not happen. But it increases expenses of bridge authorities by producing more transactions.

Updated 30/04/2018 12:55 3 Comments

Incorrect error on procedural macros with colons (`::`) in their names

intellij-rust/intellij-rust

When a procedural macro has a :: in its name, a syntax error is shown even thought the code is prefectly valid rust.

Example:

#![feature(proc_macro, specialization)]

extern crate pyo3;

use pyo3::prelude::*;

#[py::class]
struct MyClass {
    num: i32,
    debug: bool,
    token: PyToken,
}

Cargo.toml:

[package]
authors = ["konstin"]
name = "mylib"
version = "0.1.0"
crate-type = ["cdylib"]

[dependencies.pyo3]
version = "0.2.5"
features = ["extension-module"]

The following incorrect error is shown in line 7: '!' expected, got '::'

Updated 14/03/2018 20:54 1 Comments

AHMAD ALFI ADZ-DZIKRI - RTEMS/ Gentoo/ FreeBSD

gsocindonesia/coaching2018

Coach assistant: @probeadd

Sanity checks:

  • [x] GitHub profile photo + detailed profile description

Organizations:

  1. RTEMS
  2. Gentoo
  3. Fedora

Ideas:

Organization Idea Difficulty Technologies & Tools
RTEMS Improve the Raspberry Pi BSP Peripherals C
Gentoo Full Rust Support Medium Rust , Cargo , Ebuild,Bash
Free BSD Integrate MFSBSD into the release building tools Medium C,make,shell

Mentors:

  1. Gentoo - Full Rust Support:Luca Barbato lu_zero@gentoo.org (Contacted)
  2. FreeBSD =Integrate MFSBSD into the release building tools<allanjude@freebsd.org >
  3. RTEMS - Improve the Raspberry Pi BSP Peripherals Unit: Alan Cudmore email@address.com

To do: * [ ] Fix issue Ebuild and eselect Gentoo * [ ] Fix issue Rustup and Cargo

Proposals:

  1. Gentoo Proposal
  2. Organization: Proposal title
  3. Organization: Proposal title

Extras:

  • [x] Initial contribution to target organization. Link to PR: Rustup
  • [ ] Personal .github.io website & portfolio:
  • [ ] StackOverflow profile & activities:
Updated 25/03/2018 10:03 4 Comments

Store the blocks which include deployment/upgrade transactions in the contracts

poanetwork/poa-bridge

Currently the block number which includes deployment transaction stored in the database file. This number could be used in recovery scenarios to try reapplying all skipped transactions: go through all the blocks since the block the contract was created and filter deploy/withdraw events. It is also useful to deploy several new bridge instances: every bridge will start filtering events starting from the same block.

In order to simplify deployment of new bridges it could be worth to store this information in the bridge contract instead of storing it in database file: bridge deployment process could get this information from the contracts in order to generate database file with correct checked_deposit_relay, checked_withdraw_relay and checked_withdraw_confirm. So, it will be no necessity to distribute the database file among the target systems.

Proposals for changes

Solidity 1. Introduce deployedAtBlock public field in HomeBridge and ForeignBridge contracts. 2. This field initialized within the contract creation process with the block number.

Rust 1. Change the logic of generating database file as so if there are no checked_deposit_relay, checked_withdraw_relay and checked_withdraw_confirm defined in the file, deployedAtBlock is read from the corresponding contract and the parameters are initialized in the values received from the contract. 2. Remove the code which write home_deploy and foreign_deploy to the database file.

Updated 13/03/2018 07:35 2 Comments

Store gas consumption limits in the bridge contracts

poanetwork/poa-bridge

As per the bridge deployment process it is necessary to create configuration file for every bridge instance. There is a section in the configuration file which allows to define gas consumption limits which will be used in the bridge transactions: deposit_relay withdraw_confirm withdraw_relay Since these limits are common for every bridge instance it is redundant to distribute this info among all nodes before deployment begins. Moreover these limits needs to be updated in case of gas consumption changes if the bridge contract is upgraded as per #7. In order to reduce human factor it is worth to update this automatically. Some big constant value for these parameters is not good option to have - it could cause delays in validation since limits will be too big for block validators especially in the Ethereum Foundation network.

Proposal for changes

Solidity 1. Introduce gasLimitWithdrawRelay in the contract HomeBridge. 2. Introduce gasLimitDepositRelay and gasLimitWithdrawConfirm field in the contract ForeignBridge. 3. Setup this parameter as part of bridge contracts deployment process. 4. Introduce ability to change these parameters as part of bridge contract upgrade process as soon as #7 is implemented. Events must be raised in case of gas limit changes.

Rust 1. Change bridge/src/bridge/deploy.rs to setup gasLimitWithdrawRelay, gasLimitDepositRelay and gasLimitWithdrawConfirm as part of deployment process. 2. Introduce new functionality in bridge/src/bridge/deposit_relay.rs, bridge/src/bridge/withdraw_confirm.rs and bridge/src/bridge/withdraw_relay.rs in order to pickup gas limits from the corresponding contract. In order to optimize number of requests made to the blockchain nodes the limits could be re-read as soon as the corresponding event received from the bridge contracts.

Updated 13/03/2018 07:36 1 Comments

No indication if the contract executiton requires more gas than it was specified

poanetwork/poa-bridge

Steps to reproduce:

  1. Change the configuration file to reduce gas for deposit_relay to 25400.
  2. Run the bridge with RUST_LOG=debug
  3. Send ether to the HomeBridge contract

When the bridge is trying to send deposit confirmation to the ForeignBridge contract it reports that transaction was sent and deposit completed even it was not handled properly (in my case it was not included in a new block at all):

INFO:bridge::bridge::deposit_relay: got 1 new deposits to relay
DEBUG:<unknown>: [29] Calling: {"jsonrpc":"2.0","method":"eth_sendTransaction","params":[{"data":"0x26b3293f00000000000000000000000037a30534da3d53aa1867adde26e114a3161b2b1200000000
0000000000000000000000000000000000000000000a8503892e10001b254563acfa39c680314bf887d84561a8dddb3bb832664d0adaa16a2142a896","from":"0xf3ee321df87781864f46f6464e764c2827fca73b","gas":
"0x6338","gasPrice":"0x430e23400","to":"0x6500d471c8973d95493b417c44ab85f31265a2b6"}],"id":29}
INFO:bridge::bridge::deposit_relay: relaying 1 deposits
...
INFO:bridge::bridge::deposit_relay: deposit relay completed

Moreover the counter of checked_deposit_relay is increased in the database file. So, the bridge is in incorrect state after sending of such transaction.

Expected behavior

  1. The bridge must track the transactions it is sending and reports if something was wrong with them.
  2. If some relay or confirm operation cannot be completed due to failed transactions the corresponding counter in the database file must not be incremented.
Updated 28/02/2018 09:56

(Epic) Reverse bridge logic in order to do all expensive confirmations on HomeContract side

poanetwork/poa-bridge

Background

The basis of the bridge operations is events EVM sends as part of contract execution. Every bridge instance is looking for new events and as soon as an event appears after transaction applying from a new block, the bridge performs an action.

For example, when an account sends ether to the brigde contract in the Home side of the bridge (the left side), Deposit events is produced. The bridge instance catches this event and creates transaction on the Foreign (right) side to invoke deposit() method of the bridge contract. This transaction is considered as a bridge signature under the fact that some user transfers asset to the Foreign network.

<a href=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgYnJpZGdlIGRlcG9zaXQgKHNpbXBsZSBmbG93KQoKcGFydGljaXBhbnQgIkhvbWUgQWNjb3VudCIgYXMgSEEADhNzaWRlXG5Db250cmFjACEHQwA7DkIAbwVcbkluc3RhbmNlAEgFQjEAYA5Gb3JlaWduADwURkMKCkhBLT4rSEM6IGV0aGVyCm5vdGUgbGVmdCBvZiBIQwogICAgYXNzb3NpYXRlZAAKBXdpdGggdHggaWQKZW5kIG5vdGUKSEMtPi1CMTogRACBegYoc2VuZGVyLCB2YWx1ZSkAUQZvdmVyIEIxAEwIdW1lZCB0aGF0AGAFACsGID09IHJlY2lwaWVudABVCkIxLT5GQzoAglYIKAAaCQBaBywAgQoGKQo&s=rose” target=“_blank”> <img src=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgYnJpZGdlIGRlcG9zaXQgKHNpbXBsZSBmbG93KQoKcGFydGljaXBhbnQgIkhvbWUgQWNjb3VudCIgYXMgSEEADhNzaWRlXG5Db250cmFjACEHQwA7DkIAbwVcbkluc3RhbmNlAEgFQjEAYA5Gb3JlaWduADwURkMKCkhBLT4rSEM6IGV0aGVyCm5vdGUgbGVmdCBvZiBIQwogICAgYXNzb3NpYXRlZAAKBXdpdGggdHggaWQKZW5kIG5vdGUKSEMtPi1CMTogRACBegYoc2VuZGVyLCB2YWx1ZSkAUQZvdmVyIEIxAEwIdW1lZCB0aGF0AGAFACsGID09IHJlY2lwaWVudABVCkIxLT5GQzoAglYIKAAaCQBaBywAgQoGKQo&s=rose” alt=“bridge deposit (flow with 3 bridges in action)” width=“550” /> </a>

In order to achieve availability and reliability, there are more than one bridge instances (N). During these bridges configuration it is settled that just subset (M, M <= N) of them is enough to transit the state between two networks. So, if M bridges confirmed the same state transition operation (deposit), this state committed on the opposite side of the bridge:

<a href=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgYnJpZGdlIGRlcG9zaXQgKGZsb3cgIHdpcmggMwAWB3MgaW4gYWN0aW9uKQoKcGFydGljaXBhbnQgIkhvbWUgQWNjb3VudCIgYXMgSEEADhNzaWRlXG5Db250cmFjACEHQwA7DkIAgQIFXG5JbnN0YW5jZSAjMQBLBUIxAAggMgAiBjIAMCAzAEoGMwCBMw5Gb3JlaWduAIEPFEZDCgpIQS0-K0hDOiBldGhlcgpub3RlIGxlZnQgb2YgSEMKICAgIGFzc29zaWF0ZWQACgV3aXRoIHR4IGlkCmVuZCBub3RlCkhDLT5CMTogRACCXwYoc2VuZGVyLCB2YWx1ZSkAGQYyAAIdLUIzACYZAIEQBW92ZXIgQjEsIEIyLCBCMwCBEgh1bWVkIHRoYXQAgSYFAHIGID09IHJlY2lwaWVudACBGwpCMS0-K0ZDOgCEAwgoABsJAIEiBywAgVEGKQpGQy0-LUZDOiAxc3Qgc2lnbmF0dXJlIHJlY2VpdmVkCkIyABkxMm5kADUVMwBoLEZDOiAzcgA0FQCBKgkic3RhdGVcbmNvbW1pdGVkIgo&s=rose” target=“_blank”> <img src=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgYnJpZGdlIGRlcG9zaXQgKGZsb3cgIHdpcmggMwAWB3MgaW4gYWN0aW9uKQoKcGFydGljaXBhbnQgIkhvbWUgQWNjb3VudCIgYXMgSEEADhNzaWRlXG5Db250cmFjACEHQwA7DkIAgQIFXG5JbnN0YW5jZSAjMQBLBUIxAAggMgAiBjIAMCAzAEoGMwCBMw5Gb3JlaWduAIEPFEZDCgpIQS0-K0hDOiBldGhlcgpub3RlIGxlZnQgb2YgSEMKICAgIGFzc29zaWF0ZWQACgV3aXRoIHR4IGlkCmVuZCBub3RlCkhDLT5CMTogRACCXwYoc2VuZGVyLCB2YWx1ZSkAGQYyAAIdLUIzACYZAIEQBW92ZXIgQjEsIEIyLCBCMwCBEgh1bWVkIHRoYXQAgSYFAHIGID09IHJlY2lwaWVudACBGwpCMS0-K0ZDOgCEAwgoABsJAIEiBywAgVEGKQpGQy0-LUZDOiAxc3Qgc2lnbmF0dXJlIHJlY2VpdmVkCkIyABkxMm5kADUVMwBoLEZDOiAzcgA0FQCBKgkic3RhdGVcbmNvbW1pdGVkIgo&s=rose” alt=“bridge deposit (simple flow)” width=“750” /> </a>

This is a basic scenario which is enough to transite state between two EVM-based networks. The maximum (for the third bridge) gas consumption of deposit operation is about 90000.

The use case considered by parity-bridge (https://github.com/paritytech/parity-bridge, commit ceaf22f) is to connect Ethereum Foundation (Home side) and a private PoA networks (Foreign side). So, it is assumed that we need to keep as cheap as possible transactions on the Home side whereas consumption of the gas on the Foreign side does not make much sense. This is due to the reason that most probably bridge instances are managed/supported by owners of the PoA network so almost infinite resources could be spent to maintain transactions from bridges within the PoA network.

That is why the original development of parity-bridge is focused on the reduce number of transactions and gas consumption for operations made on the Home side of the brige. In order to transit the state from the Home to the Foreign network three bridges will send three transactions. And only one bridge is being chosen to perform state transition from the Foreign side to Home.

It becomes posisble owing to the architecture when confirmations of bridges to transfer the state are collected on the Foreign side (remember that it costs nothing) and as soon as needed number of confirmation gathered the bridge which sent the latest confirmation is responsible for forwarding all signatures (which are part of the confirmations) to the Home side.

<a href=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgcGFyaXR5LWJyaWRnZSB3aXRoZHJhdyBmbG93IChvcmlnaW5hbCkKCnBhcnRpY2lwYW50ICJIb21lIEFjY291bnQiIGFzIEhBAA4Tc2lkZVxuQ29udHJhYwAhB0MAOw5CAHIFXG5JbnN0YW5jZSAjMVxuKEJJICMxKQBUBUIxABEgMgAqBzIAKgcyAEIgMwBbBzMAWwczAIFODkZvcmVpZ24AgSoURgCBLw8AIwgAgX4MRkEKCm5vdGUgcmlnaHQgb2YgRkEKICAgIGJhbACBXQV0cmFuc2ZlcgAQBXNpbWlsYXIgdG8gRVJDLTIwIHRva2VuCmVuZCBub3RlCkZBLT4rRkM6ICJleHRlcm5hbAA4CVxuKHJlY2lwaWVudCwgdmFsdWUpIgBsEEMAdwVhc3Nvc2lhdGVkAIEGBXdpdGggdHggaWQAXgtDLT5CMzogVwCDewcAShIAHQYxAAIhLUIyACodQjEAgUwHc3VibWl0U2lnbmF0dXJlKHNpZywgAIFDECwAgSMGAFQHRkM6IDFzdCBzaWcgcmVjZWl2ZWQKQjMAEz4ybmQAQg8yAGk5RkM6IDNyAEEPAIQLBWxlZgCDBwxzZW5kZXIgPT0gQkkgYWRkcmVzcwCEIwVoYXNoID09AAMFAINQEQCCBQkAgyoOMjogQ29sbGVjdGVkAIJHCXMoAF4GLABOBQCDNQoABCYtQjMAMSRCMgCBbgYibWVzc2FnZSgAZwVcbnMAg00JaGFzaCwgMXN0AAMTMm5kABkTM3JkKSIAgVAJbGlzAIZCBQBJCXMAhAwZAIMzBkhDOiAiAIlBCCgALBNcbgCETRgiCkhDLT5IQTogZXRoZXIK&s=rose” target=“_blank”> <img src=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgcGFyaXR5LWJyaWRnZSB3aXRoZHJhdyBmbG93IChvcmlnaW5hbCkKCnBhcnRpY2lwYW50ICJIb21lIEFjY291bnQiIGFzIEhBAA4Tc2lkZVxuQ29udHJhYwAhB0MAOw5CAHIFXG5JbnN0YW5jZSAjMVxuKEJJICMxKQBUBUIxABEgMgAqBzIAKgcyAEIgMwBbBzMAWwczAIFODkZvcmVpZ24AgSoURgCBLw8AIwgAgX4MRkEKCm5vdGUgcmlnaHQgb2YgRkEKICAgIGJhbACBXQV0cmFuc2ZlcgAQBXNpbWlsYXIgdG8gRVJDLTIwIHRva2VuCmVuZCBub3RlCkZBLT4rRkM6ICJleHRlcm5hbAA4CVxuKHJlY2lwaWVudCwgdmFsdWUpIgBsEEMAdwVhc3Nvc2lhdGVkAIEGBXdpdGggdHggaWQAXgtDLT5CMzogVwCDewcAShIAHQYxAAIhLUIyACodQjEAgUwHc3VibWl0U2lnbmF0dXJlKHNpZywgAIFDECwAgSMGAFQHRkM6IDFzdCBzaWcgcmVjZWl2ZWQKQjMAEz4ybmQAQg8yAGk5RkM6IDNyAEEPAIQLBWxlZgCDBwxzZW5kZXIgPT0gQkkgYWRkcmVzcwCEIwVoYXNoID09AAMFAINQEQCCBQkAgyoOMjogQ29sbGVjdGVkAIJHCXMoAF4GLABOBQCDNQoABCYtQjMAMSRCMgCBbgYibWVzc2FnZSgAZwVcbnMAg00JaGFzaCwgMXN0AAMTMm5kABkTM3JkKSIAgVAJbGlzAIZCBQBJCXMAhAwZAIMzBkhDOiAiAIlBCCgALBNcbgCETRgiCkhDLT5IQTogZXRoZXIK&s=rose” alt=“bridge withdraw (flow, 3 bridges)” width=“850” /> </a>

The procedure to confirm the state transition (submitSignature) is the most expensive one. It consumes about 270000 gas (if it was sent to provide a final signature) since requires lots of mathematical operations and use the contract storage extensively.

Here is the list gas consumption for all operations performed by bridge: Handling deposit() on the Foreign side (deposit_relay) = 90861, every bridge instance Handling submitSignature() on the Foreign side (withdraw_confirm) = 265380, every bridge instance Handling withdraw() on the Home side (withdraw_relay) = 72444, one bridge instance

Relatively huge gas consumption is the only the reason why patity-bridge is not recommended to be used with Ethereum Foundation network in the right side: the meaningless from the state transaction confirmation operation will cost ~5 USD (exchange rate 1000 USD per 1 ether). But cases when a PoA network is on the Home side and the Ethereum Foundation network is on the Foreign side make sense since they allow to split transactions traffic between two networks which could have positive effect on network performance and transactions fees.

Proposal for costs optimization

From the analysis above it is evident that most costly transactions happens on the right side of the bridge. So, if Ethereum Foundation network is there so it will not be cost effective to send several transaction to confirm the deposit and several transaction to confirm the withdraw.

It means that all operations for confirmation should happens on the left side of the bridge and it equals of mirroring handling of confirmation for the current implementation: 1. As soon as a user deposits coins, validators confirmations must be gathered on the left side of the bridge. It will be called deposit_confirm. 2. When last confirmation received, one validator (one which sent the final confirmation) is responsible for forwarding the confirmation to the right side: deposit_relay. 3. If a user sends request to exchange tokens to coins, the Withdraw event is handled and every bridge instance sends the confirmation directly to the left side of the bridge. It is withdraw_relay.

<a href=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgUG9BIENUVCBicmlkZ2UgZGVwb3NpdCAoMwALB3MpCgpwYXJ0aWNpcGFudCAiSG9tZSBBY2NvdW50IiBhcyBIQQAOE3NpZGVcbkNvbnRyYWMAIQdDADsOQgBtBVxuSW5zdGFuY2UgIzFcbihCSSAjMSkAVAVCMQARIDIAKgcyACoHMgBCIDMAWwczAFsHMwCBTg5Gb3JlaWduAIEqFEYAgS8PACMIAIF-DEZBCgpIQS0-K0hDOiAiY29pbnMiCm5vdGUgcmlnaHQgb2YgSEMKICAgIGFzc29zaWF0ZWQACgV3aXRoIHR4IGlkCmVuZCBub3RlCkhDLT5CMzogRACDDAYoc2VuZGVyLCB2YWx1ZSkAGQYxAAIdLUIyADgHAC0RAIEQBW92ZXIgQjEsIEIyLCBCMwCBEQh1bWVkIHRoYXQAgSUFAHEGID09IHJlY2lwaWVudACBGgpCMQCBZgdzdWJtaXRTaWduYXR1cmUoc2lnLAAnCgCBLgcsAIFdBgCBFgdIQzogMXN0IHNpZyByZWNlaXZlZApCMwATPjJuZABCDzIAaTlIQzogM3IAQQ8AgzkVAIIaCkJJIGFkZHJlc3MAg2IFaGFzaCA9PQADBSgAgXYZAINlDjI6IENvbGxlY3RlZACCSAlzAIN2CWhhc2gAg3QKAAQmLUIzADEkQjIAgW8GIm1lc3NhZ2UoAGcFXG5zAINOCWhhc2gsIDFzdAADEzJuZAAZEzNyZCkiAIFQCWxpcwCGAQUASQlzAIQNGQCDNAZGQzogIgCJDQcoACsTXG4AhE0YIgpGQy0-LUZBOiB0b2tlbnMK&s=rose” target=“_blank”> <img src=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgUG9BIENUVCBicmlkZ2UgZGVwb3NpdCAoMwALB3MpCgpwYXJ0aWNpcGFudCAiSG9tZSBBY2NvdW50IiBhcyBIQQAOE3NpZGVcbkNvbnRyYWMAIQdDADsOQgBtBVxuSW5zdGFuY2UgIzFcbihCSSAjMSkAVAVCMQARIDIAKgcyACoHMgBCIDMAWwczAFsHMwCBTg5Gb3JlaWduAIEqFEYAgS8PACMIAIF-DEZBCgpIQS0-K0hDOiAiY29pbnMiCm5vdGUgcmlnaHQgb2YgSEMKICAgIGFzc29zaWF0ZWQACgV3aXRoIHR4IGlkCmVuZCBub3RlCkhDLT5CMzogRACDDAYoc2VuZGVyLCB2YWx1ZSkAGQYxAAIdLUIyADgHAC0RAIEQBW92ZXIgQjEsIEIyLCBCMwCBEQh1bWVkIHRoYXQAgSUFAHEGID09IHJlY2lwaWVudACBGgpCMQCBZgdzdWJtaXRTaWduYXR1cmUoc2lnLAAnCgCBLgcsAIFdBgCBFgdIQzogMXN0IHNpZyByZWNlaXZlZApCMwATPjJuZABCDzIAaTlIQzogM3IAQQ8AgzkVAIIaCkJJIGFkZHJlc3MAg2IFaGFzaCA9PQADBSgAgXYZAINlDjI6IENvbGxlY3RlZACCSAlzAIN2CWhhc2gAg3QKAAQmLUIzADEkQjIAgW8GIm1lc3NhZ2UoAGcFXG5zAINOCWhhc2gsIDFzdAADEzJuZAAZEzNyZCkiAIFQCWxpcwCGAQUASQlzAIQNGQCDNAZGQzogIgCJDQcoACsTXG4AhE0YIgpGQy0-LUZBOiB0b2tlbnMK&s=rose” alt=“PoA CTT bridge deposit (3 bridges)” width=“850” /> </a>

<a href=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgUG9BIENUVCBicmlkZ2Ugd2l0aGRyYXcgKDMADAdzKQoKcGFydGljaXBhbnQgIkhvbWUgQWNjb3VudCIgYXMgSEEADhNzaWRlXG5Db250cmFjACEHQwA7DkIAbgVcbkluc3RhbmNlICMxAEsFQjEACCAyACIGMgAwIDMASgYzAIEzDkZvcmVpZ24AgQ8URgCBFA8AIwgAgWMMRkEKCm5vdGUgbGVmdCBvZiBGQQogICAgYmFsYW5jZVxudHJhbnNmZXIKZW5kIG5vdGUKRkEtPitGQzogdG9rZW5zADAPQwA6BWFzc29zaWF0ZWQASQV3aXRoIHR4IGlkADwLQy0-QjE6IFcAgx4HKHNlbmRlciwgdmFsdWUpABoGMgACHi1CMwAnGgCBUgVvdmVyIEIxLCBCMiwgQjMAgRUIdW1lZCB0aGF0AIFoBQB0BiA9PSByZWNpcGllbnQAgWUKQjEtPitIQzoAhEUJKAAcCSxcbgCBKAUsAIFWBikKSEMtPi1IQzogMXN0IHNpZ25hdHVyZVxucmVjZWl2ZWQKQjIAMxoAgXMGADoSMm5kADYWMwAfLUhDOiAzcgA1FgAYCCJzdGF0ZVxuY29tbWl0ZWQiAIFLB0E6IGNvaW5zCg&s=rose” target=“_blank”> <img src=“https://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgUG9BIENUVCBicmlkZ2Ugd2l0aGRyYXcgKDMADAdzKQoKcGFydGljaXBhbnQgIkhvbWUgQWNjb3VudCIgYXMgSEEADhNzaWRlXG5Db250cmFjACEHQwA7DkIAbgVcbkluc3RhbmNlICMxAEsFQjEACCAyACIGMgAwIDMASgYzAIEzDkZvcmVpZ24AgQ8URgCBFA8AIwgAgWMMRkEKCm5vdGUgbGVmdCBvZiBGQQogICAgYmFsYW5jZVxudHJhbnNmZXIKZW5kIG5vdGUKRkEtPitGQzogdG9rZW5zADAPQwA6BWFzc29zaWF0ZWQASQV3aXRoIHR4IGlkADwLQy0-QjE6IFcAgx4HKHNlbmRlciwgdmFsdWUpABoGMgACHi1CMwAnGgCBUgVvdmVyIEIxLCBCMiwgQjMAgRUIdW1lZCB0aGF0AIFoBQB0BiA9PSByZWNpcGllbnQAgWUKQjEtPitIQzoAhEUJKAAcCSxcbgCBKAUsAIFWBikKSEMtPi1IQzogMXN0IHNpZ25hdHVyZVxucmVjZWl2ZWQKQjIAMxoAgXMGADoSMm5kADYWMwAfLUhDOiAzcgA1FgAYCCJzdGF0ZVxuY29tbWl0ZWQiAIFLB0E6IGNvaW5zCg&s=rose” alt=“PoA CTT bridge withdraw (3 bridges)” width=“850” /> </a>

Changes required:

  1. Optimization of the ForeignBridge contract:
    • remove submitSignature(), signature() and message();
    • remove the corresponding maps keeping information about signatures;
    • rework deposit() to work with list of signatures similarly to withdraw() of the original HomeBridge contract;
    • consider to have the code to cover costs of the bridge (isMessageValueSufficientToCoverRelay(), getWithdrawRelayCost() and estimatedGasCostOfWithdraw)
  2. Modification of the HomeBridge contract:
    • rework withdraw() to accept signatures directly from bridge validators similar to deposit() method on the original ForeignBridge contract.
    • there is no need any more to cover costs of the bridge on the Home side so isMessageValueSufficientToCoverRelay(), getWithdrawRelayCost() and estimatedGasCostOfWithdraw must be removed.
    • introduce submitSignature(), signature() and message() and corresponding maps to keep information about signatures similarly to the original ForeignBridge contract.
  3. Rework bridge/src/bridge/withdraw_relay.rs to send withdraw() to the HomeBridge contract as soon as Withdraw event appeared on the Foreign side. Every bridge instance should invoke this method simiarly deposit() of the original HomeBridge contract.
  4. Rework bridge/src/bridge/deposit_relay.rs to wait for CollectedSignatures and forward deposit with the list of signatures to the ForeignBridge contract.
  5. Rename bridge/src/bridge/withdraw_confirm.rs to bridge/src/bridge/deposit_confirm.rs and modify it as so Deposit event is being waited from Home side and submitSignature() is invoked by every bridge instance. The corresponding changes must be made in bridge/src/bridge/mod.rs in order to run the confirmation module.
  6. Modify bridge/src/database.rs and bridge/src/bridge/mod.rs to not track checked_withdraw_confirm in the database and update checked_deposit_confirm instead.
  7. Modify bridge/src/config.rs to
    • get rid of options transaction.withdraw_confirm.gas, transaction.withdraw_confirm.gas_price.
    • introduce options transaction.deposit_confirm.gas, transaction.withdraw_deposit.gas_price.
Updated 23/05/2018 17:05 2 Comments

[Sapling] Specify and implement the signature scheme for spend authorization

zcash/zcash

We intend to use an EdDSA signature on the Jubjub curve to authorize spends. The original Ed25519 scheme made some assumptions might not be applicable to the Jubjub curve.

Generalised EdDSA is specified in the paper EdDSA for more curves, and standardized as RFC 8032. It requires a hash function with output size 2b bits, where 2<sup>b-1</sup> is greater than the field size. For Jubjub, b = 256, so the obvious choice is BLAKE2b. For the “pure” version of EdDSA that doesn’t prehash the message, this would be called PureEdJubjub-BLAKE2b. The RFC specifies additional constraints on the input to the hash function (the dom{2,4} byte and context) that are not present in the paper.

To close this ticket, * [x] choose all 11 parameters mentioned in the paper * [x] decide how to resolve any incompatibility between the RFC and the paper * [x] nail down options for verification (strictness of parsing, whether cofactorless verification is used, whether low-order points are rejected) * [x] add PureEdJubjub-BLAKE2b to the protocol spec * [ ] implement it * [ ] publish test vectors (including signatures rejected for various reasons)

Updated 23/04/2018 19:29 6 Comments

Use ciphersuite strings to select ciphers?

miscreant/xstream

The test vectors presently use the following ciphersuite strings:

  • XSTREAM_X25519_HKDF_SHA256_AES128_SIV
    • Key Agreement: X25519
    • KDF: HMAC-SHA-256
    • Symmetric Cipher: AES-128-SIV
  • XSTREAM_X25519_HKDF_SHA256_AES128_PMAC_SIV
    • Key Agreement: X25519
    • KDF: HMAC-SHA-256
    • Symmetric Cipher: AES-128-PMAC-SIV

However, none of the existing XSTREAM implementations accept these as arguments, but instead take an AES-SIV versus AES-PMAC-SIV string which is passed directly to Miscreant (where applicable).

For the Rust implementation in particular, it would be nice to have object safe traits for STREAM which allow us to use either STREAM or XSTREAM via a trait object, and in particular to select the stream encryptor type to use based on a string.

Updated 02/04/2018 00:52 3 Comments

Style Guide

SCAII/SCAII

We should have a style guide for the whole project, in most case just delegating to other style guides.

I don’t know where we’d put this, probably not in this repository but maybe in the website+dev docs someday if we get that.

Rust:

  • The semi-official guide https://github.com/rust-lang-nursery/fmt-rfcs/blob/master/guide/guide.md
  • rustfmt with default settings exits without changing anything (these two guidelines end up being roughly equivalent in practice).
  • unsafe blocks are allowed, but must be audited before being pulled in.

Python:

  • Pylint exits cleanly (this is tentative until I work with pylint more, it’s overly restrictive so we’d probably end up messing with this until we have a “default” .pylintrc we’re happy with.
  • This also generally extends to following PEP8
  • Use autopep8 for autoformatting.
  • Python 3, don’t bother supporting 2
  • Use PEP484 annotations ??

Javascript:

??? I don’t know JS but I feel like we should have SOMETHING.

JSHint maybe? http://jshint.com/

Protobuf:

  • Fields may be declared required and optional/required distinctions are important.
  • Prefer ``` oneof foo { A a = 1; B b = 2; }

message A {} message B { required type field = 1; } ```

to

enum Foo {
    A = 0;
    B = 1;
}

required Foo foo = 1;

// only set if Foo is B
optional type field = 2;

General:

  • We need notes like “if your code is going to interact with Javascript like Viz, avoid integers over MAX_SAFE_INTEGER where possible” and such.
  • All functions and modules should be documented, except potentially test suites.
  • All languages and plugins should use semantic versioning 2.0.
    • Different pieces need not be in tandem (e.g. the glue version need not match the core version).
    • The whole project has a version number independent of the version of its components. That said, it still follows semantic versioning (that is, if any of its components have a certain piece of their versioning bumped, the overall project must AT MINIMUM have that bumped. So if core has to up its major version number because of an API break, so much the entire project).
      • This applies to official plugins such as the RTS.
      • The exception to this is test binaries

Git:

  • Generally this: https://github.com/agis/git-style-guide
  • Feature branches are considered volatile and should not have work based on them (that is, they may be freely rebased, squashed, etc), except in special cases (i.e. two people working on coinciding features may synchronize their work and then merge/rebase before merging into dev. This is how @jedirv and I should have handled the viz/test suite thing).
  • If an important hotfix must be immediately added to master, all branches must immediately rebase to apply this change (that is, dev should be rebased and features should be rebased on top of the new dev etc).
  • Master is sacrosanct and its history must not be overwritten
  • Dev is semi-sacrosanct and history generally should not be overwritten except in special cases.
  • Branches should always be rebased onto the target branch before merge

Recommendations

The following are just recommendations for contributors and such, anything may be used as long as the end result conforms to guidelines: * Anaconda Python for Python * VS Code with the RLS extension for Rust * Chrome/Chromium and Firefox for JS and web in general. * Prost for Rust protobuf; regular protoc compiler for Python/JS * Compiling JS protobuf into a single file rather than multiple.

Updated 24/03/2018 05:12

Send adaptive program queries to Agent

SCAII/SCAII

We need to be able to query the agent for feedback on a game state during replays for explanation stuff. This will be evolving a lot, my first intuition is that we should provide some simple visualization primitives (more high level than what the backend uses) like “pie chart”, “heatmap”, “highlight entity” etc. We need to take care to not reinvent an entire plotting package.

Since we’re using JS, something like bokeh is out, but we may be able to find some nice JS plotting libraries to lighten the load. (Which is a shame, it’d be nice to send rendered graphs from bokeh, they look quite nice).

Depends, minimally on #14, #16, #17, #18 + some API and protobuf consideration

Updated 24/03/2018 05:11 2 Comments

Port zcash-fetch-params to rust

zcash/zcash

I’m creating a seperate issue to follow up on the discussion with @str4d from #2597:

Since zcash seems to be into rust, is there interest to have those scripts translated? I can see why shell might be preferable, but I could offer porting them if you give me some instructions on how you would like to have it integrated into the repo. (I’d prefer to handle this in a followup pull request though).

Interesting idea. Are you envisaging that we build a binary for downloading parameters instead of using a script? It would certainly be more portable, and it would also be a self-contained chunk of code which would make integration easier. We are still figuring out how exactly we will integrate Rust into Zcash, but a separate binary like this would likely be done in a different way to rewriting or extending internal zcashd code. You should open an issue for discussing this!

This is basically what I’m suggesting. :)

From the top of my head, I can think of the following: - Add it to librustzcash and export a function that can be called from zcashd with C++ - cargo init a small project in a subfolder, compile it during the build and install the binary as zcash-fetch-params. If the build system changes to cargo, this can be moved to src/bin/ and cargo is going to build it as one of the resulting binaries.

Updated 16/04/2018 21:59

Rust: Support external crates

mesonbuild/meson

I was told that support of Rust was experimental so I can expect hickups and that’s understandable but currently it seems external crates are not supported. Since you can’t really do much in Rust without external crates, it basically means that Meson doesn’t really support Rust. IMO until this issue is addressed in some way, support for Rust shouldn’t be advertised at all.

Updated 21/03/2018 21:01 31 Comments

smallstack

vendethiel/projects

Still needs to be heavily refactored. How to encode number of instruction in type / have function arities depend on that?

Updated 06/03/2018 09:48 1 Comments

Fork me on GitHub