Contribute to Open Source. Search issue labels to find the right project for you!

rgw: continuation of the auth rework -- AWSv4


This PR is the third part of the authentication & authorisation rework. It’s focused on extending support for the AWSv4 schema across S3 auth engines.

Many low-level primitives have been dissected as-is from the current code and just encapsulated behind new interfaces. Potential clean-ups (like switching from std::string to boost::string_ref) will be pushed to this PR in separated commits.

The branch has been verified with s3-tests (both in AWSv2 and AWSv4), Tempest and AWS Java SDK (the AWSv4’s streaming mode:

TODO: * more testing: Keystone & AWSv4, Browser Upload & AWSv4, … * clean-ups.

Depends on: #11179, #14432.

CC: @yehudasa, @mattbenjamin.

Updated 30/04/2017 15:48

DupeGuru 4.0.3 thinks that a newer copy of a file is the original


Dear Virgil,

first of all, a big thank-you for releasing this valuable piece of software into the sphere of freedom!

I have recently instigated that DupeGuru become part of the MX-16 project. MX-16 is an improved Debian not relying heavily on systemd, and it is becoming more and more popular. One of the project’s repo men thankworthily built a package from your 4.0.3 source code for the MX-Test Repo, and I’m presently testing it. You can see the discussion on it here:

However, upon testing DupeGuru, it seems to me that it has a very severe bug enthreating its very function. I’m not sure if it really is a bug or rather a configuration issue, or some strange kind of behaviour that is related to MX-16 (and MX-16’s dealing with timestamps).

What I did was this:

I did a test run on a friend’s historically grown data.

My friend has a folder on his HDD which he once created when he had to travel abroad. The pretext is: He copied one folder (let’s call it “the original folder”) to another folder before he travelled abroad, and then he copied that folder to a USB stick (which he took with him) before travelling in order to not loose his data in case the device he left at home gets stolen.

So my friend has one prominent duplicate folder on his HDD with duplicate files in it. The original folder in question was created on March 8th, 2017. The copy of this folder was created approximately two hours later (at least that’s what nemo tells me).

So you should think that DupeGuru would regard an original file in the original folder as being the original, and prefer to save the original and to delete the duplicate copy file (otherwise the application you made probably wouldn’t be called “DupeGuru”). However, what DupeGuru actually does it is to offer the original for deletion, while not offering the (newer) copy to be deleted (you can’t check it for deletion in the GUI).

Is this a bug, or a configuration issue, or an MX-16 issue?

As you can see in the discussion, one of the MX-16 developers guesses that DupeGuru maybe cannot read timestamps, whereas I couldn’t quite believe that.

Greetings, Joe

Updated 30/04/2017 16:02 1 Comments

User Controller


User controller


  • showLogin(container) - show the login form view
    • container - the container to display the view in
  • login(params) - handle submit of the login user form

    • params - holds the form inputs
  • showLogout(container) - show the logout form view

    • container - the container to display the view in
  • logout() - handle submit of the logout form
Updated 30/04/2017 15:02

Support datapackages as data sources


[Implementation suggestion]

Detection - one of: - format == datapackage - basename(url) == datapackage.json

Resolution: When format is datapackage, url is resolved according to data-package-identifier spec

Resource selection: - An optional resource parameter. - If numeric: specifies the index of the resource - If string: specifies the name of the resource (first one that matches) - If unspecified: takes the 1st resource

Resource handling: - If Tabular data package - iterates on the resource itself, using the schema - If Non-Tabular datapackage - resolves to the file referenced by the resource, and continues regular handling from there.

Updated 30/04/2017 14:52

Idea: country recycling taxonomy


It seems each country has it’s own recycling approach. On packaging we see often different instructions for different countries. We have not yet found a good way to transcribe these instructions into the packaging-field. And the instructions are always the same.

Seems a good idea to create a recycling taxonomy. This taxonomy should list all the countries, per country the recycling possibilities and per possibility what can be recycled. The latter should refer to the packaging taxonomy.

If needed a country can be subdivided in regions or smaller entities.

The required info can be gathered from the packaging.

Updated 30/04/2017 16:49 2 Comments

implemented Event::FilesDropped for Windows


Disclaimer : this feature is not ready to be merged yet

Related forum thread:

I have implemented this feature for personal use. I write simple map editors and other small tools with the aid of SFML because I don’t want to bring out the big guns like Qt. For something like a map editor, the drag and drop functionality is very handy, especially for textures and materials.

As can be seen in the linked forum thread, I am not the only one wanting this feature. Therefore, I went ahead and created a pull request. This feature is not complete yet but it already works on Windows. I figured that once I take the first step maybe others are more inclined to chime in and collaborate.

Progress: - [x] Modify sf::Event to store the relevant information - [x] Implement interface in sf::Window (see setFileDroppingEnabled) - [x] Implement backend for Windows - [ ] Implement backend for *nix - [ ] Implement backend for OSX - [x] Add an example or another kind of test

Issues: - sf::Event::FileDropEvent is inevitably part of a union, so I couldn’t use a vector but had to resort to a raw pointer. Personally, I think this is not a big deal because the usage is still very straight forward: for (int i=0; i<event.droppedFiles.count; ++i) event.droppedFiles.files[i]; But maybe there’s a better solution? - As sf::Event::FileDropEvent only stores pointers but not the filenames themselves, their lifetime is bound to the window, not to the event. Is there any case in which this window-specific buffer can cause trouble? - Am I right in assuming that WindowImplWin32::processEvent doesn’t need to be thread-safe? Otherwise the aforementioned buffer would need to be protected by a mutex, ensuring that no concurrent accesses are possible.

Unfortunately, I’m not versed well enough in X11 to implement this feature for Linux, but I have found this which might be handy for anyone wanting to implement this feature.

Updated 30/04/2017 15:02

Do the db filter at lib level


This PR references #20

A new module valid-dbs is defined in /lib/valid-dbs.js to handle the logic of the valid dbs. It exports: - dbs An object that contains the valid db options as keys and descriptions. - InvalidDbError Custom Error to throw in functions that require a valid dbs - printDbs A function that pretty prints the dbs following this format key (description) and separated by new lines.

This also modifies the search function in main ncbi lib file to throw an exception is the db is invalid. Minor modifications to the cli file to use the valid-dbs module to handle the logic when no valid db is provided. Updates the package.json file so all the test files are run in the CI and the coverage reporter uses all the test results.

Updated 30/04/2017 14:52 1 Comments

Feature: Allow sorting of file list



  • One clicks on ‘File Name’ category -> a arrow goes down -> sorts by alphabetical order. Arrow up -> vice-versa
  • One clicks on ‘File Size’ category -> sorts from smallest to largest size. Arrow up -> vice-versa. (…)

This works like normal file sorting in a directory.

This allows the user to sort the list of files based on their preference.

Updated 30/04/2017 15:34

Feature: Some extra GUI design changes.


In the base windows (see picture below):

  • Change the orange (+) next to the bucket to a green (+) with a white center (same fill-text color as the Download button).
  • Fill the refresh button color to the solid blue like the Edit button instead of the greenish color.

Doing this makes everything more neutral instead of it appearing to be a candy shop. It also blends in nicely with the Storj colors (the green and the blue) which gives a similar experience between using the storjshare GUI and this upload program. Basically having a monotone style across the Storj apps gives for a more smooth experience.


Updated 30/04/2017 15:38

Feature: Automatically scroll down as the shards download/upload to keep-up with the process.


Right now if you start a download/download it starts downloading shard 1-x, after shard 7 the new shards that are downloaded/uploaded are our of the viewing area. By automatically scrolling down to always have the latest active shard in the viewing area helps the user to track the entire upload process without having scroll down every time to see the latest active shard.

Updated 30/04/2017 12:39

[Workflow] Add transition completed event

Branch? master
Bug fix? no
New feature? yes
BC breaks? no

Because the “entered” event is the only event dispatched after the new marking is applied, and publish’s an event upon entering into a “Place” (as opposed to completing a transition), it is not sufficient for a lot of use cases and is causing bugs.

Example: Enabled Transitions: 1. A -> B 2. B -> C 3. C -> B

Transition 1 and transition 3, will dispatch an “entered” event on Place B, forcing post transition behaviour to be the same for both transition 1 and 3.

A user might need different behaviour depending on the transition, rather the the destination. A concrete use case would be when applying an “undo” transition to a subject. One may or may not want to re-trigger all the events associated with the original transition to that Place.

I propose adding a “completed” event (ie. Transition completed) in addition to the entered event.

Updated 30/04/2017 16:25 2 Comments

Automatic generation of Menus in Sensor Data logger


In the present scenario most sensors have several configuration options, and these are defined in a dict element called ‘Params’ in each sensor’s class. The Sensor data Logger application uses this dictionary to autogenerate menus with the ‘key’ as the name , and corresponding ‘values’ as a submenu . When the user opens a menu and clicks on a ‘value’ , the ‘value’ is passed to a function whose name is the corresponsing key , and which must be defined in the sensor’s class. An Example to illustrate The following Params dictionary defined in the class of MPU6050 creates a menu as shown in the second image self.params = { 'powerUp':['Go'], 'setGyroRange':[250,500,1000,2000], 'setAccelRange':[2,4,8,16], 'KalmanFilter':[.01,.1,1,10,100,1000,10000,'OFF'] } |mpu6050|

As shown in the image , when the user clicks on ‘8’ , MPU6050.setAccelRange(8) is executed.

There are obvious limitations to this approach: - For functions with no arguments, a dummy variable is defined just so the user has something to click on.
- This does not work for functions which require an arbitrary user defined input. e.g. , the Kalman filter menu shows a range of pre-defined values, wherein it should actually be a QDoubleSpinBox input.

I propose the following changes: - For functions with no arguments, the value field should be None , and the GUI should generate a QMenuAction instead of a QSubMenu. - For functions with variable input arguments, the menu should display a QSlider/QSpinBox/QTextArea/QColorDialog etc . For such functions, instead of providing a list of values, a string mentioning the type of input must be declared. Possible types : integer,double,string etc.

Updated 30/04/2017 11:18

Add text drawing


Currently, Pixel doesn’t support drawing text. This needs to be changed. Implementing text drawing in Go is actually way simpler than I thought it would be.

First of all, there’s an awesome package, which implements standard font face interface, which is well designed and very useful. Subpackages implement several font types and implements TrueType font loading with support of the standard font face interface. So we’ve got all we need.

Now, the design of text drawing in Pixel.

Text drawing should be done in a not-yet-existing "" package. This package should export (at least) one type: Text. Creating one should be done by text.New(face), where face is a font.Face value. The *Text value returned from New will use that specific font face.

Text type should implement io.Writer interface, so drawing text to it is as simple as fmt.Sprintf(txt, "hello world"). Printing to Text draws text “to the Text object”. To draw the printed text to another target (such as a window, canvas, batch, …), simply use txt.Draw(target). Clearing needs to be supported too (txt.Clear()).

Drawing text needs to be fast. There should be no problem drawing a full screen of random text using a tiny font face at 60 FPS without a performance hit. This is something that many other libraries fail to accomplish, but it’s actually not very hard. It’s easily achievable by caching glyphs.

So, to recap the interface.

  • ""
    • New(font.Face) *Text
    • type Text
      • Clear()
      • Draw(pixel.Target)
      • Write(b []byte) (n int, err error)

This is the basis for the interface. More methods will probably be added, such as WriteString, or SetDot (the position of the text line).

Updated 30/04/2017 14:15 7 Comments

Add text input from window


Currently, *pixelgl.Window supports checking key and mouse presses. Implementing a user-editable text is close to impossible to do correctly just using this functionality. GLFW library provides a way to correctly get a text input from the user, which depends on their keyboard settings and all that stuff.

Pixel should support this functionality too, through a simple interface. I suggest this single method. - (*pixelgl.Window).Typed() string - returns a string of characters the user typed on the keyboard since the last call to (*pixelgl.Window).Update

Note, that the Typed method does not and should not handle text deletion. That’s easy to implement with (*pixelgl.Window).Pressed or (*pixelgl.Window).JustPressed method.

Updated 30/04/2017 10:04

Add window icon support


Add a *pixelgl.Window method to set the icon of the window. The icon is usually displayed at the top bar of the window, or in the task bar of the desktop environment. - (*pixelgl.Window).SetIcon(pixel.Picture) - sets the icon of the window to the given picture

I don’t think a getter is necessary.

Also, make it possible to add an icon to pixelgl.WindowConfig by adding a Icon field there. Is the (*pixelgl.Window).SetIcon actually necessary with this field?

Updated 30/04/2017 15:32 2 Comments

Changing hostname in confconsole doesn't change postfix hostname


As reported by @Dude4Linux:

Just tried the new hostname feature of confconsole. It made the expected changes in /etc/hostname and /etc/hosts, but left the parameter ‘myhostname’ in postfix set to the value set during installation, in this case ‘core’. Not sure if this is a feature or a bug. Would users expect that all references to hostname set during install would be changed, or should a warning be issued re: the need to manually configure apps such as apache2 that make use of the hostname.

Updated 30/04/2017 10:04 1 Comments

Reflection. Support Type.GetInterfaceMap(Type)


Returns an interface mapping for the specified interface type: cs [System.Runtime.InteropServices.ComVisible(true)] public virtual System.Reflection.InterfaceMapping GetInterfaceMap (Type interfaceType);

Updated 30/04/2017 09:40

Add cursor hiding


Add two *pixelgl.Window methods: - (*pixelgl.Window).SetCursorVisible(bool) - sets the visibility of the mouse cursor inside the window client area - (*pixelgl.Window).CursorVisible() bool - returns the visibility status of the mouse cursor

Updated 30/04/2017 09:39

Queuing jobs for Irwin


Irwin can take significantly more load now. It has been averaging ~16% CPU utilisation over the past week, so it should be able to be loaded with ~6x the amount of requests that it’s been given.

Expanding Irwin so be able to reprocess users in the report queue, checking perf leaderboards, and tournament leaders seems like a natural progression to defend lichess from cheaters.

The simplest approach in my mind is to put the top 50-100 from each standard perf, the top 10 from each standard tournament, and the players in the report queue into a single list. Sort them by when they were last checked by Irwin, and randomly provide one of the names in the top 10 of that list to Irwin (players who were updated longest ago).

Updated 30/04/2017 13:12

[Gang] The Hit Contract: The Non-Conversion Based Snowball Method.


:cl: Cobby add: Influence can now be earned fairly rapidly by placing hits on people with the new hit contract. You get extra points if the target is a gang member, and even more if they are a gang head! balance: Anyone who is a target of a hit cannot be converted by said gang. balance: Contracts are tied to the gang that spawned them. If you are using another gang’s contract, you can rack up points for them very easily. DON’T DO THIS! Just hide it from them, as they just lost 40 Influence for nothing. /:cl:

WARNING: This has NOT been tested to the fullest extent since it relies on multiple people being on at once. This was also “tested” in the morning hours so take that with a grain of salt.


The Hit contract is a fairly high cost item/investment that allows for Rapid acquisition of points by killing particular people. It trades potential converts into blood-stained influence, with bonuses if they are in enemy gangs. Because of this, a new viable strategy is to rush tagging, save for the hit contract, then play ping pong and rack up points to give your entire [albeit small] gang tons of weaponry and ammo.

Targets are mind-based and won’t appear again in the list of potential targets after a hit was placed on them by your gang, nor allow for conversion. This is to prevent exploits such as killing a non-gang target , cloning and repeat; killing a gang member, cloning, then penning them; or simply penning someone who you have a hit on. There are no tells for others if there’s a hit on someone sans examining the document, so you as the gang leader have to coordinate with your team or you could get them outted as a leader.

The contract automatically checks to see if the target has died once a hit has been placed. It does not have to be at the gang’s hands specifically, leaving room for beneficial accidents.

Points Overview

Cost - 40 Target Is not a gang member or gets Deleted (bombs or badmins, or both!) - 50 [10 profit] Target is a Gang Member, but not a boss - 65 [25 profit] Target is a Gang Leader - 75 [35 profit]

As you can see, this always gives a full refund on a successful hit. This is not actual paper and will cost inventory slots, so you may have to hide the contracts later leaving room for a smart detective to stop you before you get too powerful.


This was made as a way to deter conversion, which seemed to be a problem from the gang feedback thread. While you definitely want early conversion, mid-late game will now reward you for playing less conversion snowball [aka take 20 years for a gang round to end] and more aggressive PRECISE KILLING [if you murderbone you lose potential points] since conversion lowers your point potential while also increasing the point potential of others. Apart from that, you now have 2 general approaches you can take to gang: The Conversion ZergRush OR The High Octane Hit Squad, both being viable.

Updated 30/04/2017 16:29 9 Comments

Switch off / on domains

  • domain always shows small on/off button in bottom LHS
  • when you hit ‘off’, you immediately go to a static HTML page saying ‘this site is switched off’
  • every time you then try to access that domain, you redirect immediately to that static HTML page
  • to switch site back ‘on’, there is a slider on the static HTML page. slider design is such that user must click the slider circle and drag it all the way to the right-hand-side in order to switch the site back on
  • should be possible for this entire feature to be switched on / off via Nudge options
  • should be possible for certain domains to be always ‘off’ by default when you start a new Chrome session
Updated 30/04/2017 08:59

Vote to skip?


The current playlist is planned hours in advance. I do think that occasionally playing a complete album is nicer than purely global shuffle, but it does mean we can’t really have a request form like r/a/dio. The next best thing is a link to vote to skip an upcoming track.


  1. Add an AJAX-powered button to send a “skip track” request to the backend.
  2. Keep track of number of listeners of the channel.
    • Maybe it should only be listeners who are listening via, rather than or other sites. Those other people won’t be voting.
    • The number of listeners on can be measured by logging unique IPs over a short time.
  3. Keep track of number of skip votes.
  4. Remove the track from the playlist when the skip votes reach a certain proportion of the listeners.
Updated 30/04/2017 11:15 2 Comments

New track/album request form


There’s a feedback thread on /music/, but something directly on the site would be nice as well. Maybe this would work best as a generic comment form, with suggested topics (eg “music you want”).


  1. Implement a pretty form as an HTML page.
  2. Implement the backend functionality. The upload_voice function should be helpful.
  3. Link from the radio page.
Updated 30/04/2017 08:30

Feature: Provide a way to change the color of the mention


Mentioned names are hardcoded to #ff0000.

It would be good to be able to change that maybe, via ACP or something. As some styles might not look good with that color, or some user’s may just want a more subtle appearance (like not changing the font color from the rest of the text, and just keeping it bold).

Updated 30/04/2017 08:07 1 Comments

Stop focusing automatically when opening the terminal in a DOM node


When using open to expose an xterm.js terminal in the DOM, the terminal gets focus automatically.

This may be useful for very simple use cases (like the demo of xterm.js), but it can make it difficult for more complex applications to control where the focus is.

The best thing to do here is stop focusing on the terminal inside open and let the user decide when to focus.

Initiated from #640.

Updated 30/04/2017 07:40

Notification should contain the topic/post title


Can the notification message also mention the post? e.g.,

“You have been mentioned by username in "Title of the post”

I think it’d be super helpful to know where you’ve been mentioned… You know, in case it’s a topic you want to respond to or one you might want to totally ignore. :)

Updated 30/04/2017 08:10

Feature request: iterate over flattened features


Turf has a flatten method that creates a feature collection. It would be helpful to have a flattenEach method that traverses them with a callback without allocating them and consuming memory. I can submit a PR if there is interest in this feature.

Note, this would likely introduce helpers as a dependency. The helpers module in turn has no dependencies. So, this shouldn’t be a problem unless it is desired to keep all these low level modules free of any dependencies.

Updated 30/04/2017 16:28 2 Comments

Add Flip Filter

Branch? 1.0
Bug fix? no
New feature? yes
BC breaks? no
Deprecations? no
Tests pass? yes
Fixed tickets #918
License MIT
Doc PR #920

This pull request adds a simple filter implementation to allow for image flipping (both along the x and y axis). Included are tests and documentation, as well as some general cleanup of the file.

The included OptionsResolver abstraction was introduced to aid in the API differences between Symfony =2.3 and >=2.7. My long-term intention is to move other filter loader implementations to using the options resolver as well; once this has been done, this abstraction can be removed in the 2.x branch where it is no longer required without affecting the method calls within filters.

Updated 30/04/2017 00:19

Adds debian package and install instructions


Ich habe gerade der HSR\connect auf Debian 9.0 (stretch) und Netrunner (Debian KDE fork) getestet. Die Installation von der Ubuntu Quelle schlägt leider mit dem folgenden Fehler fehl. Beide Distributionen zeigen beim Kommando ‘apt update’ den gleichen, folgenden Fehler an:

E: The repository ‘ stretch Release’ does not have a Release file. N: Updating from such a repository can’t be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details.

Ich gehe davon aus, dass APT kein passendes Packet in der Quelle findet und desshalb den Update abbricht. Ich habe deshalb das Ubuntu Verzeichnis kopiert und commited in mein eigenes Repository. Ich werde gleich einen Pull-Request erstellen, damit ihr es mal ansehen könnt. Ich wäre froh, wenn ihr dann das Package zeitlich mal hochladet, ich möchte ihn auch nutzen 😃.

Updated 30/04/2017 10:01 1 Comments

Implement editor for Command permission overrides


This will be implemented bot-side: #50

  • This should be a separate config page: Button to go there would be on the server-select page.
  • It would work very similar to the way the config page works, where it would simply iterate through all the commands specified in the json config (storage/app/configFile.json iirc) and always display just a name of a command, and a drop-down menu containing standard permissions, and a list of blocked channels…
    • Default = -1,
    • Everyone = 0,
    • Nobody,
    • ServerOwner,
    • Admins,
    • Moderators,
    • SubModerators,
    • Members
  • Default values are obvious.
  • This configuration would be stored similar to the standard config, in the ServerID folder as commands.json
Updated 29/04/2017 21:03

V2 event rework


This is the big one.

Adding in the idea of a “Session”.

It’s single user, multi-person. We can send invitations, reminders, people can cancel and confirm via sms, etc.

Essentially, it rips up the whole “V2::” system and replaces it with a smaller footprint, largely because it leaves the scheduling to the users and the people, and then manages the reminding and notifications to the app.

It’s largely test free at the moment, which will need to change, but not untill this sprint is done, I’m afraid.

Updated 29/04/2017 20:59

Implement editor for Custom Commands.

  • Custom Commands are stored in the server config as an array of CustomCommand objects.
  • The editor will display this array on the left side, and CustomCommand structure on the right side.
  • List (left side) - Display all the elements, where the user can edit them by clicking on them (this will display the element on the right side); add a new ones by hitting an add button, and delete some by clicking either tiny x button in the list, or selecting an element and clicking a delete button.
  • Editor window (right side) - Display properties of selected or new CustomCommand, these currently are:
    • string ID
    • string Description
    • string Response - Bigger text field than standard text input.
    • long[] RoleWhitelist - List of roles that can use this command. Use our standard role-picker
    • bool DeleteRequest
Updated 29/04/2017 21:10


  • [ ] Implement localisation system.
  • [ ] Move all the original strings into English
  • [ ] French
  • [ ] Italian
  • [ ] German
  • [ ] Spanish
  • [ ] Custom language support EFIGS would be the must-have for any localisation, but we can have more…
Updated 29/04/2017 20:43

Fork me on GitHub