Skip to content

[ENH, WIP] Adding CMI test#12

Draft
adam2392 wants to merge 1 commit intopy-why:mainfrom
adam2392:cmi
Draft

[ENH, WIP] Adding CMI test#12
adam2392 wants to merge 1 commit intopy-why:mainfrom
adam2392:cmi

Conversation

@adam2392
Copy link
Copy Markdown
Collaborator

@adam2392 adam2392 commented Apr 21, 2023

Towards: #5

Changes proposed in this pull request:

  • CMI based CI test

This is still a draft.

Before submitting

  • I've read and followed all steps in the Making a pull request
    section of the CONTRIBUTING docs.
  • I've updated or added any relevant docstrings following the syntax described in the
    Writing docstrings section of the CONTRIBUTING docs.
  • If this PR fixes a bug, I've added a test that will fail without my fix.
  • If this PR adds a new feature, I've added tests that sufficiently cover my new functionality.
  • I have added a changelog entry succintly describing the change in this PR in the whats_new relevant version document.

After submitting

  • All GitHub Actions jobs for my pull request have passed.

Signed-off-by: Adam Li <adam2392@gmail.com>
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 21, 2023

Codecov Report

❌ Patch coverage is 3.91061% with 172 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@138539c). Learn more about missing BASE report.

Files with missing lines Patch % Lines
pywhy_stats/cmi.py 0.00% 149 Missing ⚠️
pywhy_stats/monte_carlo.py 23.33% 23 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main      #12   +/-   ##
=======================================
  Coverage        ?   22.08%           
=======================================
  Files           ?        6           
  Lines           ?      240           
  Branches        ?       31           
=======================================
  Hits            ?       53           
  Misses          ?      186           
  Partials        ?        1           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread pywhy_stats/cmi.py
n_shuffle: int = 100,
random_seed: Optional[int] = None
) -> PValueResult:
rng = np.random.default_rng(random_seed)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning this?

Comment thread pywhy_stats/cmi.py
n_shuffle: int = 100,
random_seed: Optional[int] = None
) -> PValueResult:
rng = np.random.default_rng(random_seed)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

returning?

Comment thread pywhy_stats/cmi.py
val = hxyz - (hxz + hyz - hz).mean()
return val

def _preprocess_data(self, data: pd.DataFrame) -> pd.DataFrame:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_, indices = nbrs.kneighbors(z_arr)
elif method == "kdtree":
tree_xyz = scipy.spatial.cKDTree(z_arr)
indices = tree_xyz.query(z_arr, k=k, p=np.inf, eps=0.0, workers=n_jobs)[1].astype(np.int32)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

else: return ValueError

rng = np.random.default_rng(seed=random_seed)

# initialize the final permutation order
restricted_perm = np.zeros((n_samples,))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
restricted_perm = np.zeros((n_samples,))
restricted_perm = np.zeros((n_samples,), dtype=np.int32)

if it has not been used to permute another sample. If it has been, then the
algorithm looks at the next nearest-neighbor and so on. If all k-nearest
neighbors of a sample has been checked, then a random neighbor is chosen. In this
manner, the algorithm tries to perform permutation without replacement, but
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

citation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants