Skip to content

add non-channel query iterator method #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Nov 25, 2016
Merged

Conversation

kevina
Copy link
Contributor

@kevina kevina commented Nov 15, 2016

Continuation of #53

@kevina kevina force-pushed the kevina/faster-query branch from 0e7f1bb to 5dd9e04 Compare November 22, 2016 22:44
@kevina kevina force-pushed the kevina/faster-query branch from 5dd9e04 to dc3eeed Compare November 22, 2016 23:40
@kevina kevina changed the title WIP: add non-channel query iterator method add non-channel query iterator method Nov 22, 2016
@whyrusleeping
Copy link
Member

@Kubuxu could you help CR this one?

Copy link
Member

@whyrusleeping whyrusleeping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, the 'having to switch back to legacy mode' kinda sucks. Maybe theres a better way of doing that? Not a huge deal though.

@whyrusleeping
Copy link
Member

Maybe on conversion to legacy we swap out the next function in case the user calls NextSync again?

@kevina
Copy link
Contributor Author

kevina commented Nov 23, 2016

@whyrusleeping I confused, what are you getting at. If Next() is called and then NextSync() do you want the NextSync() method to go through legacyResults instead? Or do you want NextSync() to error out?

@whyrusleeping
Copy link
Member

Lets just have NextSync go through the legacy method instead. That way we don't accidentally miss an entry thats trying to be sent on the legacy channel.

@kevina
Copy link
Contributor Author

kevina commented Nov 23, 2016

Okay will do.

@kevina
Copy link
Contributor Author

kevina commented Nov 23, 2016

@Kubuxu @whyrusleeping one thing I was unsure of is the handling of the automatic closing in NextSync(). Even if there is not a go process, there might still be something that needs to be closed, for example the leveldb iterator should probably be closed when done to help free up resources. Comments/reviews on this aspect will be appreciated. Also see ipfs/go-ds-leveldb#1 for some context.

@whyrusleeping
Copy link
Member

@kevina I'm thinking we should require that the user always call Close on the query when they are done with it.

@kevina
Copy link
Contributor Author

kevina commented Nov 23, 2016

@whyrusleeping okay I cleaned up the logic of when legacyResults is used and when close() is called.

This still needs testing.

@whyrusleeping
Copy link
Member

@kevina sounds good, ping me when you've added more tests then

@kevina
Copy link
Contributor Author

kevina commented Nov 23, 2016

@whyrusleeping Unit tests added.

I will do a manual integration test tomorrow, but other than that it should be good to go.

@kevina
Copy link
Contributor Author

kevina commented Nov 23, 2016

@kevina I'm thinking we should require that the user always call Close on the query when they are done with it.

From IRC:

kevina: whyrusleeping: in order to match the interface it seams NextSync() should call the close function if it is defined
whyrusleeping: okay, so if NextSync returns 'false' we assume things have been closed?
kevina: I think, that is what we have to do to match the interface.
kevina: In go-ds-leveldb i.Release() will be called when it has no more results to send in runQuery(), see https://github.com/ipfs/go-ds-leveldb/blob/a37289361537b8109b1a4ec3c7f194c5d2b39414/datastore.go#L141
kevina: when not using a channel the close function passes in will serve to call i.Release() see https://github.com/ipfs/go-ds-leveldb/blob/a37289361537b8109b1a4ec3c7f194c5d2b39414/datastore.go#L108
kevina: basically I think I figured this out :)
whyrusleeping: Okay, that makes sense

@kevina kevina force-pushed the kevina/faster-query branch from e5e9df7 to 83a8d8c Compare November 23, 2016 07:51

go b.Process.CloseAfterChildren()

r.legacyResults = b.Results().(*results)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this assertion always safe?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes based on the code for ResultBuilder.Results() in the same file.

@kevina
Copy link
Contributor Author

kevina commented Nov 23, 2016

@whyrusleeping All tests pass when using this version with go-ipfs. I would call this good to go.

@whyrusleeping
Copy link
Member

@kevina Cool, Would just like to have @Kubuxu and/or @lgierth review this first

@whyrusleeping whyrusleeping merged commit 63c28bc into master Nov 25, 2016
@whyrusleeping whyrusleeping deleted the kevina/faster-query branch November 25, 2016 08:41
@whyrusleeping whyrusleeping removed the status/in-progress In progress label Nov 25, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants