Skip to content
This repository was archived by the owner on Oct 30, 2023. It is now read-only.

GIRAPH-1185#69

Closed
yukselakinci wants to merge 2 commits intoapache:trunkfrom
yukselakinci:phase22
Closed

GIRAPH-1185#69
yukselakinci wants to merge 2 commits intoapache:trunkfrom
yukselakinci:phase22

Conversation

@yukselakinci
Copy link
Copy Markdown
Contributor

Second PR for serialization. There is one more.

*
* @param <T> Object type
*/
public class KryoSimpleWrapper<T> implements Writable {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the difference between this and KryoWritableWrapper, when should we use one and when the other?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added more documentation now. The usage of KryoSimpleWrapper is similar to KryoWritabeWrapper. KryoSimpleWrapper can be used if the object being serialized is not recursive or nested, because it uses a version of kryo object that doesn't track references, hence significantly faster.

* This serialization does not support recursive/nested structures
* to provide faster serialization.
*/
public abstract class KryoSimpleWritable implements KryoIgnoreWritable {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great to switch to Java 8, to be able to keep this an interface :-)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree :-)

// non-kryo serialization.
/** Reusable Input object */
private final InputChunked input = new InputChunked(4096);
private InputChunked input;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we separate two Kryo implementations in two classes since they have a lot of differences?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we can refactor the existing code to have a better code readability. I have added a TODO for that.

* no recursive/nested objects should be serialized.
* @return Hadoop kryo which doesn't track objects.
*/
public static HadoopKryo getNontrackingKryo() {
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will someone using kryo know which implementation to use? Can we add some guidance around that?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added more documentation.

Copy link
Copy Markdown

@majakabiljo majakabiljo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline, @yukselakinci will in a separate diff refactor HadoopKryo.

@asfgit asfgit closed this in 60752aa Apr 12, 2018
@yukselakinci yukselakinci deleted the phase22 branch April 17, 2018 16:45
dlogothetis pushed a commit to dlogothetis/giraph that referenced this pull request Apr 26, 2018
dlogothetis pushed a commit to dlogothetis/giraph that referenced this pull request Apr 26, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants