Contribute to Open Source. Search issue labels to find the right project for you!

Pin lock UI feedback

ipfs/go-ipfs

<!– Output From ipfs version --all) –>

Version information: 0.4.7

<!– Bug, Feature, Enhancement, Etc –>

Type: enhancement

<!– from P0 “Critical” to P5 “Relatively Unimportant”) –>

Priority: P2

Description:

Pinning operations take the pin lock, so that only one pinning operation can run at a time. When running ipfs pin add and another operation is currently running, it’ll sit and wait until it acquires the lock and only then start to run. During this time there’s no UI feedback.

Instead it should show something like “Waiting for other pin operations to finish… ok”.

I think apart from ipfs pin add, this also applies to ipfs pin rm and ipfs add.

<!– This is for you! Please read, and then delete this text before posting it. The go-ipfs issues are only for bug reports and directly actionable features. Read https://github.com/ipfs/community/blob/master/contributing.md#reporting-issues if your issue doesn’t fit either of those categories. Read https://github.com/ipfs/go-ipfs/blob/master/docs/github-issue-guide.md if you are not sure how to fill in this issue. –>

Updated 21/03/2017 05:34 1 Comments

Repo is growing and growing

ipfs/go-ipfs

Version information:

go-ipfs version: 0.4.5-
Repo version: 5
System version: amd64/darwin
Golang version: go1.7.4

Type: Bug

Priority: P3

Description:

When creating a hashchain of objects (so each object links to a previous object), it seems repository gets much larger than expected. And it also just continues to grow and grow.

Reproduction: https://github.com/mitar/ipfs-create-chain

I started with empty repository (not sure why it is not completely empty, because I initialized it with -e):

$ date ; ./ipfs repo stat
Wed Mar 15 03:00:10 PDT 2017
NumObjects   5
RepoSize     334034
RepoPath     /Users/user/.ipfs
Version      fs-repo@5

After adding 10000 objects of a hashchain:

$ date ; ./ipfs repo stat
Wed Mar 15 03:01:16 PDT 2017
NumObjects   10005
RepoSize     2142370
RepoPath     /Users/user/.ipfs
Version      fs-repo@5

The last object in this hashchain is saying that the whole size is 1008672. 2142370 is more than twice as much.

After pinning the hashchain size jumps:

$ date ; ./ipfs repo stat
Wed Mar 15 03:01:24 PDT 2017
NumObjects   10267
RepoSize     5637553
RepoPath     /Users/user/.ipfs
Version      fs-repo@5

But leaving it running for 10 minutes just grows the repository:

$ date ; ./ipfs repo stat
Wed Mar 15 03:13:36 PDT 2017
NumObjects   10267
RepoSize     7167765
RepoPath     /Users/user/.ipfs
Version      fs-repo@5

After few more minutes:

$ date ; ./ipfs repo stat
Wed Mar 15 03:35:51 PDT 2017
NumObjects   10267
RepoSize     7452579
RepoPath     /Users/user/.ipfs
Version      fs-repo@5

And after few more hours:

$ date ; ./ipfs repo stat
Wed Mar 15 11:46:51 PDT 2017
NumObjects   10267
RepoSize     18612219
RepoPath     /Users/user/.ipfs
Version      fs-repo@5

What is happening here? This is a lot more than 1008672. OK, the list of all pins is probably around 1 MB more. So I would expect the repo to be around 2 MB.

There was no other use of the IPFS & daemon after the run of the initial script. I just left it running.

Updated 17/04/2017 18:20 15 Comments

(re)Start properly crediting package authors on synocommunity.com

SynoCommunity/spksrc

The author field on the package view on synocommunity.com/package/$package is set to the uploader by default (not the Maintainer in INFO, as those don’t necessarily have the same value). Over time, that means that a lot of authors aren’t properly credited for their work. So, let’s fix that.

How it works: - Make sure you use your Github account name in the package Maintainer field; - Use your Github account name to register on synocommunity.com/register; - Tell me that you’ve registered, so we can link the two.

Started with some people here: https://github.com/SynoCommunity/spksrc/pull/2643#issuecomment-278968908 I’ll have to start comparing packages with the original PR’s for the rest…

Updated 24/02/2017 00:18 15 Comments

More granular address announcements

ipfs/go-ipfs

Sometimes a go-ipfs listener can not directly bind to its external address. This is e.g. the case with NAT, where it binds to an IP address in the local network, while other nodes dial it using the local router’s public IP address. We jump through all kinds of hoops to work around that, and one of these is “observed addresses”, where other nodes tell us which addresses they’ve seen us using.

Another example are HTTP-based transports like /ws, which will in many cases be behind an SSL-terminating reverse proxy. Bound address: /ws, external address: /wss. There is a way to discover that we’re behind a reverse proxy (by passing a special HTTP header), but that would only work after the first request has come in. Until then, the node would be unaware of its /wss address.

We could just as well have an ExternalAddresses config option though (or ideally both). This might also be useful for announcing /dns addresses. The default bootstrappers will eventually be bootstrap.libp2p.io or so, and they’d be able to announce it.

Updated 25/04/2017 00:04 8 Comments

Broadwell - Problem adding packages.synocommunity.com repository to rs3617xs+

SynoCommunity/spksrc

Hello, it seems there is a problem adding packages.synocommunity.com repository to rs3617xs+ - it reports “Invalid location”. The same procedure on RS2211 works well.

Here are the http request and reply sniffed from the network … (packages.synocommunity.com was resolved to 104.31.94.179)

GET /?package_update_channel=beta&unique=synology_broadwell_rs3617xs%2B&build=8451&language=csy&major=6&arch=broadwell&minor=0&timezone=Amsterdam HTTP/1.1
Host: packages.synocommunity.com
User-Agent: synology_broadwell_rs3617xs+ DSM6.0-8451 Update 2 (package)
Accept: */*
HTTP/1.1 422 UNPROCESSABLE ENTITY
Date: Wed, 02 Nov 2016 11:47:48 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d1523eb3770083bbc6dad219f752ed3ab1478087268; expires=Thu, 02-Nov-17 11:47:48 GMT; path=/; domain=.synocommunity.com; HttpOnly
Server: cloudflare-nginx
CF-RAY: 2fb75a9440222348-FRA

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>422 Unprocessable Entity</title>
<h1>Unprocessable Entity</h1>
<p>The request was well-formed but was unable to be followed due to semantic errors.</p>

(and one more POST request using the same parameters)

Updated 04/04/2017 19:30 13 Comments

GC improvements

ipfs/go-ipfs

From https://github.com/ipfs/infrastructure/commit/63887a6697f17eb8f932ab4dfec2524c9e7318cd#commitcomment-14734054 in the context of ipfs/infrastructure#125

@jbenet added a note a day ago

do we need the period? isnt the watermark enough?

@lgierth commented on 63887a6 a day ago

This is an old commit – it ended up as 20m in master: e177f10

Periodic GC is run regardless of the GCPeriod option being present, from what I understand: https://github.com/rht/go-ipfs/blob/48a33ffb673fb3eaf6d20cd7ca6bed04426a2abb/cmd/ipfs/daemon.go#L492-> L507

We need periodic GC because for now only ipfs cat maybe-runs GC.

@rht commented on 63887a6 a day ago

The default value for GCPeriod is set here.

We need periodic GC because for now only ipfs cat maybe-runs GC.

The latter because surveying the entire repo for each core.Resolve is too expensive.

@jbenet commented on 63887a6 12 hours ago

The latter because surveying the entire repo for each core.Resolve is too expensive.

It should definitely happen on writes.

Also, we should have a way of not running GC again in the case that: - above gc watermark - runs gc once - gc does not drop storage below watermark - another write happens (no pinset removes) - running gc again will waste a ton of read time - another write should not trigger gc

MaybeGC could track when new unpinned things could happen: - set a dirty bit when things removed from pinset (things become cached) - set a dirty bit when things new writes without adding pins (new things cached) - maybeGC skips GC if dirty bit not set

Updated 06/03/2017 08:11 1 Comments

Fork me on GitHub